Write your first OneAgent plugin¶

Set up your environment¶

To develop custom plugins for OneAgent, you need to begin by setting up your environment.

Prerequisites:¶

1. OneAgent installed and running on your system
2. Python 3.5
3. oneagent-sdk python package installed

Obtaining OneAgent SDK

OneAgent SDK can be downloaded from web UI. Go to Settings > Monitored Technologies > Custom plugins.

Installing OneAgent SDK

The OneAgent SDK is shipped as a ZIP archive that contains this documentation and a wheel package (WHL format). You can install the SDK wheel package using pip, for example:

pip install oneagent_sdk-1.98.0.20160601.192338-py3-none-any.whl

If you need help installing Python packages, please reference the Python Packaging User Guide

Note

If you're short on time, or get stuck, the plugin source code is available in examples/demo_plugin directory in your SDK package

Your first plugin¶

The first plugin you'll write gathers data from a simple Python web application. The application itself is provided as part of the sdk package and can be run by typing:

python -m oneagent_sdk.demo_app

When run without parameters, the application listens on localhost:8769, and responds to HTTP requests with a small JSON document that looks like this:

{"counter": 147, "random": 907}

counter increases with every hit to the page, while random is a random number.

You can list the parameters of the demo application using:

python -m oneagent_sdk.demo_app --help

This includes the parameters for changing the default port and turning on authentication.

Let's take a look at the plugin. Essentially it contains 2 files: plugin.json which includes the metadata required to run the plugin (including name and version) and the Python code, which is placed in the demo_plugin.py file. Let's go through the py file first:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

import requests import json import logging from ruxit.api.base_plugin import BasePlugin from ruxit.api.snapshot import pgi_name class DemoPlugin(BasePlugin): def query(self, **kwargs): pgi = self.find_single_process_group(pgi_name('oneagent_sdk.demo_app')) pgi_id = pgi.group_instance_id stats_url = "http://localhost:8769" stats = json.loads(requests.get(stats_url).content.decode()) self.results_builder.absolute(key='random', value=stats['random'], entity_id=pgi_id) self.results_builder.relative(key='counter', value=stats['counter'], entity_id=pgi_id)

The file defines one class DemoPlugin, which in turn defines one method query. When your plugin runs, this method is called each minute to collect any data and send the data to Dynatrace Cluster Node. This particular query method does the following:

        pgi = self.find_single_process_group(pgi_name('oneagent_sdk.demo_app'))
        pgi_id = pgi.group_instance_id

All measurements in OneAgent must specify the ID of the entity that they're associated with. In short, process snapshot is a data structure containing a mapping between running processes and their IDs as assigned by OneAgent. In order to send data we need to search the snapshot. We use the find_single_process_group() method for that.

Learn more about exploring process snapshots.

        stats_url = "http://localhost:8769"
        stats = json.loads(requests.get(stats_url).content.decode())

Once we acquire the ID (additionally making sure that our demo application is running and has been detected by OneAgent) we need to gather the stats. This is easily achieved with the help of a requests package.

        self.results_builder.absolute(key='random', value=stats['random'], entity_id=pgi_id)
        self.results_builder.relative(key='counter', value=stats['counter'], entity_id=pgi_id)

Once we have both the data to report and the relevant entity IDs, we can handle it through the results_builder object. ResultsBuilder stores and processes all data sent to it (i.e., when you submit measurements as relative, it automatically computes the difference between current and previous values and only sends the difference. This is useful for reporting metrics that are increasing most of the time, but you're only interested in the rate of change, and not the absolute value.

That's it for the Python part of the plugin. Note that plugins are not limited to a single source file, but for the sake of simplicity, the demo plugin only includes a single file.

The other vital part of each custom plugin is the plugin.json file

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

{ "name": "custom.python.demo_plugin", "version": "1.1", "type": "python", "entity": "PROCESS_GROUP_INSTANCE", "processTypeNames": ["PYTHON"], "source": { "package": "demo_plugin", "className": "DemoPlugin", "install_requires": ["requests>=2.6.0"], "activation": "Singleton" }, "metrics": [ { "timeseries": { "key": "random", "unit": "Count" } }, { "timeseries": { "key": "counter", "unit": "Count" } } ] }

Let's go through the contents of the file.

  "name": "custom.python.demo_plugin",
  "version": "1.1",
  "type": "python",

There are a few simple properties at the beginning: plugin name, version`, and ``type. plugin name and version are fairly self explanatory. type describes the plugin technology, which for custom plugins is always "python".

  "entity": "PROCESS_GROUP_INSTANCE",
  "processTypeNames": ["PYTHON"],

entity and processTypeNames are used to inform OneAgent as to when it should execute the plugin. In this case the values state that the plugin should be executed when the process groups identified by Python (process type 29) are detected.

  "source": {
    "package": "demo_plugin",
    "className": "DemoPlugin",
    "install_requires": ["requests>=2.6.0"],
    "activation": "Singleton"
  },

The source section covers information specific for a given type of plugin, in this case Python. Plugins written in Python, don't necessarily monitor Python. The first two lines list the package name and class that will be imported and executed by OneAgent. They correspond to the name of the Python file that contains the code and the name of the class defined in it.

The property install_requires lists the plugin's dependency, the requests library. Other libraries, along with their version requirements, can be listed here. If you're familiar with the Python packaging ecosystem, you'll note that these properties are similar to what you would write in your setup.py or requirements.txt files when developing a Python package.

The last property activation informs is an additional parameter that controls the plugin activation process. It means that only one plugin class instance will be created, no matter how many process groups of type python are detected on the system.

  "metrics": [
    {
      "timeseries": {
        "key": "random",
        "unit": "Count"
      }
    },
    {
      "timeseries": {
        "key": "counter",
        "unit": "Count"
      }
    }
  ]
}

Lastly, the metrics section describes the data that's gathered by the plugin. This section provides two metrics that mirror both what our demo application serves, and also what the Python code collects.

Plugin simulator¶

To facilitate plugin development, this SDK comes with a tool that can execute your plugin without the need for you to run OneAgent or connect to Dynatrace Cluster Node. This plugin simulator is a purely Python solution that's designed to mimic OneAgent, with a few limitations:

• The process snapshot must be provided manually.
• Plugin configuration (if any) needs to be provided manually.
• No data is sent to or received from Dynatrace Cluster Node.
• The Python environment contains the libraries required by the plugin.

The format chosen for both snapshot and properties configuration is JSON. Here is an example simulator_snapshot.json file that is sufficient for triggering the demo plugin:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

{ "entries": [ { "process_type": 29, "group_name": "oneagent_sdk.demo_app", "processes": [ { "process_name": "python3.5", "properties": { "CmdLine": "-m oneagent_sdk.demo_app", "WorkDir": "/home/demo", "ListeningPorts": "8769" } } ] } ] }

The structure of simulator_snapshot.json resembles ProcessSnapshot. The fields that are not required at the moment, or that can be generated by the simulator (for example, process group IDs), can be omitted. In this particular snapshot you can see one process group of type 29 (Python) with one process attached. This should be enough to run the demo plugin, provided that the demo_app is started.

The simulator is started with the command:

oneagent_simulate_plugin

Or the shorter:

oneagent_sim

No additional arguments are needed if the command is run from the directory that contains the plugin. For other options, please refer to the command help message.

The simulator runs until it is stopped manually (usually by pressing Ctrl-C), output is written to the terminal, and a short summary of operations is provided upon completion, which should look something like this:

Plugin simulator summary:
At 14:23:52 reported measurements:
[PluginMeasurement(monitoredEntityId=1, tsIdentifier='custom.python.demo_plugin:random', dimensions=[], value=56.0, aggregation=1)]
At 14:23:52 reported status: OK
At 14:24:52 reported measurements:
[PluginMeasurement(monitoredEntityId=1, tsIdentifier='custom.python.demo_plugin:counter', dimensions=[], value=1.0, aggregation=1),
 PluginMeasurement(monitoredEntityId=1, tsIdentifier='custom.python.demo_plugin:random', dimensions=[], value=313.0, aggregation=1)]
At 14:25:52 reported measurements:
[PluginMeasurement(monitoredEntityId=1, tsIdentifier='custom.python.demo_plugin:counter', dimensions=[], value=1.0, aggregation=1),
 PluginMeasurement(monitoredEntityId=1, tsIdentifier='custom.python.demo_plugin:random', dimensions=[], value=567.0, aggregation=1)]

If an error occurs, the summary may offer a hint as to why:

At 14:28:30 reported status: UNINITIALIZED
At 14:28:30 reported status: ERROR_UNKNOWN
Status details:
Traceback (most recent call last):
  File "C:\work\envs\plugins_dev_35\lib\site-packages\requests\packages\urllib3\connection.py", line 137, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "C:\work\envs\plugins_dev_35\lib\site-packages\requests\packages\urllib3\util\connection.py", line 91, in create_connection
    raise err
  File "C:\work\envs\plugins_dev_35\lib\site-packages\requests\packages\urllib3\util\connection.py", line 81, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

In this example, the error was caused by not having a running oneagent_sdk.demo_app process. So the plugin was unable to gather data.

The report at the end of simulator run is not the only source of information. You can also analyze log messages that are sent out while the simulator runs.

Run the plugin¶

By now, you should have a grasp of what the demo plugin can do, but you haven't run it yet!. It's super easy; only 3 steps are required:

1. Build the plugin.
2. Upload the plugin to Dynatrace Cluster Node.
3. Restart the plugin.

oneagent_build_plugin

Starting oneagent_build_plugin
Validating plugin.json against schema
Plugin data: Plugin name=custom.python.demo_plugin, version=1.0
...
========================================
Plugin deployed successfully into /opt/dynatrace/oneagent/plugin_development/custom.python.demo_plugin
Plugin archive available at /opt/dynatrace/oneagent/plugin_development/custom.python.demo_plugin.zip
========================================

oneagent_build_plugin --help

Upload the plugin¶

Go to Settings > Monitored technologies > Custom plugins. Use "Upload plugin" button. Select the ZIP file you created with oneagent_build_plugin.

The plugin should be uploaded in staging mode and visible on the same Monitored technologies page.

Alternatively, the oneagent_upload_plugin commandline tool may be used to perform the upload. In order to use it however, you must first acquire an authentication token.

The token is generated via the web UI using the Generate key button, which is located next to the Upload plugin button (see above). The token may than be stored in a local file as an environmental variable or passed directly to the upload tool:

oneagent_upload_plugin -t pEnLDvdMTA2lhJ_IUNC8U

If the upload is successful, you should see an output resembling this:

Starting new HTTPS connection (1): demo.live.ruxit.com
plugin has been uploaded successfully

To learn more about the commandline upload, please type:

oneagent_upload_plugin --help

Once the plugin is successfully uploaded, you may decide to promote the plugin to production mode by clicking the plugin name and clicking the Upgrade now button.

Restart the agent¶

The last step is to restart OneAgent, so that it can load and run the new plugin. oneagent_build_plugin command will perform this step automatically. Depending on your operating system, restarting OneAgent can be done in a few ways. On Ubuntu, type:

sudo service oneagent restart

On Windows, go to Service Manager and restart the OneAgent service, or type:

net stop "Dynatrace OneAgent" && net start "Dynatrace OneAgent"

To verify that the plugin works, make sure that the demo_app is running. Navigate to the app's host details page (see below), find the host you're working on and click the oneagent_sdk.demo_app process.

From there, click Further details.

Here, you can see the metrics:

Congratulations, you've written your first OneAgent plugin! If you've run into issues, please consult the troubleshooting guide

Next steps

The capabilities of OneAgent plugins don't end with monitoring demo applications. To learn more, please explore these additional resources:

• Adding configuration to your plugin.
• Error handling for your plugin.
• Monitoring multiple applications with your plugin.
• More information on plugin simulator
• More information on plugin.json files.
• Plugin lifecycle.
• Technical documentation.