Cloud storage for profiling reports
If you are using the Team plan for Sciagraph, you can also use Sciagraph to profile automated jobs, in particular in production. Since the jobs are running automatically, perhaps in a container or some other ephemeral runtime location, this raises the question of where you should store reports for later retrieval. In order to reduce the configuration burden of running Sciagraph in production, the Team plan also includes cloud storage for reports.
This howto covers:
- How to find download instructions.
- How to install the report-viewing tool.
Automatic cloud storage when using the Team plan
Whenever a job being profiled by Sciagraph finishes, it is automatically uploaded in encrypted form to the Sciagraph cloud server, so you can retrieve it later even if your runtime environment goes away.
Sciagraph then writes out a standard Python
logging log message that will tell you how to download the report.
See the article on security and privacy of reports to learn the security implications of this design, and your responsibilities if you want to keep your profiling reports private.
Reports are also stored locally, as with profiling in development.
If you’ve overridden the default
logging destination globally, the log message will go wherever you’ve directed log messages.
Otherwise, by default log messages in Python go to stdout.
Either way, you will have a normal log message somewhere at the end of your logs that will tell you how to download the report. For example, here’s an example log message:
Successfully uploaded the Sciagraph profiling report. Job start time: 2022-03-02T10:17:36+00:00 Job ID: Unknown To see the resulting profiling report, run the following on Linux/Windows/macOS, Python 3.7+. If you're inside a virtualenv: pip install --upgrade sciagraph-report Otherwise: pip install --user --upgrade sciagraph-report Then: python -m sciagraph_report download 476e2b1a-8c3b-4c25-bff7-b69860b1200a LiEQsqO7U/Q9ygeOESWL3ekWT9zVniTtgHOBqEO2xXjSKLiGfLCH
If you just follow those instructions on your computer, you’ll end up with the report open in a browser.
By default, then, to see the profiling report for a batch job you just go look at that job’s logs; the download instructions should be one of the last messages in the log.
sciagraph-report: more details
In order to view profiling reports that you haven’t stored yourself, you will need to install the
The tool should run on Linux, macOS, and Windows, and requires Python 3.7 or later.
In general, you can install it by doing
pip install sciagraph-report.
More specifically, you can do so in a number of ways.
- If you have a virtual environment created with
python -m venv, or
conda env, you can install the package into that environment by simply running
pip install sciagraph-reportinside the activated virtualenv.
- You can install as a “user” install:
pip install --user sciagraph-report. This installs the package in a user-specific location. Newer versions of
pipwill do user installs automatically when running outside of a virtualenv; older versions will require an explicit
- You can install
pipxtool, which is designed for isolated installs of Python command-line tools.
In theory you can also install it globally to the whole machine (
sudo pip install ... on Linux or macOS), but since you’ll be installing on your development machine this is a bad idea.
In general, you should never
sudo pip install.
If you get a
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv. that means you tried to do
pip install --user inside a virtualenv.
pip install sciagraph-report, omitting the