Table of Contents
Building a Device Mapper
Read Time: 11 mins
So, you want to contribute a device mapper to the Sonar poller. Great! Here’s a walk through on how to get started.
First, you’ll need a copy of the source code. Head over to our repository and fork the poller.
Now let’s step through how to build a new mapper, and test it easily. All the code you’ll want to look at is in the src directory.
We’re going to mainly focus on DeviceIdentifiers and DeviceMappers, but for reference, here’s whats in each folder if you want to explore or work on something more complex.
- Exceptions contains any custom exceptions used by the poller.
- functions.php contains non-class based functions that can be used anywhere.
- Helpers contains classes that don’t really fit anywhere else. For example, there is a SysInfo class in here that figures out the number of processors on a poller to determine how many workers there should be in a pool. Realistically, this could probably live in Services.
- Models contains all the data models for the poller, such as Device, which represents a specific device to be monitored, or MonitoringTemplate which represents a monitoring template defined in Sonar.
- Overrides contains classes that override other third party package classes. For example, we extend the AmPHP platform using a class in here. It’s unlikely you’ll ever want to touch this.
- Pipelines contains classes that move data from one part of the system to another. For example, DeviceFactory takes the JSON data from Sonar and formats it into Device models, as well as figuring out which devices need to be polled via ICMP or SNMP.
- Services are for classes that do stuff in the context of a particular process flow. For example, there’s a Formatter in here that formats data, and an SnmpClient that performs SNMP queries.
- Tasks contain work that is run inside an AmPHP pool.
- Web contains all the logic to expose the poller web interface.
- Worker contains our custom code for the AmPHP worker. You almost certainly don’t want to mess with this either.
Building a new Device Mapper
Let’s circle back to building a new device mapper. I mentioned at the start that we’d focus on mappers and identifiers. To get identifiers out of the way, the only time you need a device identifier is if a vendor has multiple types of devices that all respond with the same response to a system.sysObjectID query (OID 1.3.6.1.2.1.1.2.0.) If they do, check out the existing device identifiers to see how they work.
Essentially, you’ll need to build a device identifier to use some other method of figuring out what the actual device is and then return the appropriate mapper based on your investigation. For example, the Ubiquiti device identifier looks at the interface names on a device to try to determine what it is.
Generally speaking, if you can not use a device identifier, you should not. It adds more queries to the mix, and delays monitoring more, but if there’s no other choice, build one.
Device Mappers
Getting back to device mappers, building a new one is fairly simple. Building a new device mapper means extending the BaseDeviceMapper class, so let’s start there. Device mappers always live inside a directory named after the manufacturer, so let’s imagine we’re making a mapper for an imaginary manufacturer like MySuperCoolRouters. First, I’m going to make a folder named MySuperCoolRouters in the DeviceMappers directory.
Next, I’m going to make a class for each type of device I want to support. Let’s imagine MySuperCoolRouters makes a super cool router called TheCoolestRouterEver. To support this device with a custom mapper, I’m going to make a new class called TheCoolestRouterEver, and extend the BaseDeviceMapper class.
If you take a look at the BaseDeviceMapper class, you’ll see that data is always passed into this class during an SNMP polling cycle, and that the base mapper will perform various queries to determine things like the interfaces on the device. The important things to notice are that you always have the Device model available to you inside your class (as $this->model) and an array of NetworkInterface objects available as $this->interfaces. Check out the NetworkInterface model to see all the data available in it — this is where we’ll likely do most of our data injection.
Setting up your mapper
Your mapper will be instantiated if a device returns a response to the system.sysObjectID query that is run inside SnmpGet that matches a mapper definition in devices.json.
The devices.json file lives in the config directory, and looks something like this:
{
"devices": [
{
"response": "1.3.6.1.4.1.161",
"device": "Poller\\DeviceMappers\\Cambium\\CanopyPMPAccessPoint"
},
{
"response": "1.3.6.1.4.1.2736.1.1",
"device": "Poller\\DeviceMappers\\Etherwan\\EtherwanSwitch"
},
{
"response": "1.3.6.1.4.1.10002.1",
"device": "Poller\\DeviceIdentifiers\\Ubiquiti"
},
{
"response": "1.3.6.1.4.1.14988.1",
"device": "Poller\\DeviceMappers\\MikroTik\\MikroTik"
},
{
"response": "1.3.6.1.4.1.17713.5",
"device": "Poller\\DeviceMappers\\Cambium\\PTP500Backhaul"
},
{
"response": "1.3.6.1.4.1.17713.6",
"device": "Poller\\DeviceMappers\\Cambium\\PTP600Backhaul"
},
{
"response": "1.3.6.1.4.1.17713.7",
"device": "Poller\\DeviceMappers\\Cambium\\PTP650Backhaul"
},
{
"response": "1.3.6.1.4.1.17713.8",
"device": "Poller\\DeviceMappers\\Cambium\\PTP800Backhaul"
},
{
"response": "1.3.6.1.4.1.17713.9",
"device": "Poller\\DeviceMappers\\Cambium\\PTP700Backhaul"
},
{
"response": "1.3.6.1.4.1.17713.11",
"device": "Poller\\DeviceMappers\\Cambium\\PTP670Backhaul"
},
{
"response": "1.3.6.1.4.1.17713.21",
"device": "Poller\\DeviceMappers\\Cambium\\EPMPAccessPoint"
},
{
"response": "1.3.6.1.4.1.17713.250",
"device": "Poller\\DeviceMappers\\Cambium\\PTP250Backhaul"
},
{
"response": "1.3.6.1.4.1.41112.1.4",
"device": "Poller\\DeviceMappers\\Ubiquiti\\AirMaxAccessPoint"
},
{
"response": "1.3.6.1.4.1.43356.1.1.1",
"device": "Poller\\DeviceMappers\\Mimosa\\BxBackhaul"
},
{
"response": "1.3.6.1.4.1.43356.1.1.2",
"device": "Poller\\DeviceMappers\\Mimosa\\BxBackhaul"
},
{
"response": "1.3.6.1.4.1.43356.1.1.3",
"device": "Poller\\DeviceMappers\\Mimosa\\AxAccessPoint"
},
{
"response": "1.3.6.1.4.1.46242",
"device": "Poller\\DeviceMappers\\Netonix\\Ws6Mini"
}
]
}To add your poller to the mix, add a new object to this file, with the appropriate response and the namespace of your new device mapper. Note the double forward slashes in here — they are needed for escaping.
Note that the response here doesn’t need to be exact. If you’re building a mapper that is valid for 1.2.3.4, 1.2.3.5, and 1.2.3.6, you can enter 1.2.3 as the response here, and as long as there isn’t a more specific response listed, the poller will use your mapper for anything starting 1.2.3.
Now it’s time to add logic to your mapper!
Mapper Logic
When your mapper is instantiated, it will be passed a Device object into the constructor, which is handled by the BaseDeviceMapper. Next, the map function will be called, passing in an SnmpResult object. We want to pass this offer to the underlying base mapper to run before we do any work on it, as the base mapper will do all the heavy lifting of fetching interfaces and other data for us. To do this, let’s call the parent map function inside our new mapper.
This gets us an SnmpResult object that will have the interfaces array populated with valid NetworkInterface objects. From this point, you can look at other device mappers in the repository to see typical things that are done to add more data here, but I’ll continue to work through this fictional example. Let’s say, for example, that TheCoolestRouterEver is some kind of aggregation device that customers are connected to, and via the SNMP OID 1.2.3.4.5.6, it returns a list of the MAC addresses that are connected to interface eth0, which is where customers are always connected. What we want to do then, is query that OID, fetch the MAC addresses, and attach them to the eth0 network interface, so that Sonar can use that data for the parent/child system, and Sonar Pulse.
Let’s get started by adding a new function to our class called getAttachedCustomers.
Now, inside this function, we’re going to run an SNMP query to get this list. We can do this by accessing the SnmpClient class that’s available through the Device object, which is exposed through the base mapper.
Assuming no exception is thrown here, the $result variable will be an SnmpResponse object. We can call getAll() on this object to get an associative array, where the key is the OID, and the value is the response.
As mentioned earlier in this article, in this hypothetical situation, we know we want to attach the results of this query to the eth0 interface. We need to iterate the interfaces in the $this->interfaces array to find the eth0 interface, and then attach the MAC addresses given by this SNMP query to it.
To do this, you can just iterate the $this->interfaces array, calling getName() on each object until you find eth0. Remember, the contents of this array are NetworkInterface objects, so you can check that class for other methods.
Next, we are going to call getConnectedLayer1Macs on the network interface. When you want to attach customers to a network interface, we set them as Layer1 connected, regardless of the layer they are connected to. Layer1 always takes precedence when Sonar calculates relationships for Pulse and the parent/child system, so if we know these customers are connected here, it’s best and safest to attach them this way.
Next, we simply add the results of our SNMP query to the $existingMacs array, and set it back on the interface. Finally, we update the NetworkInterface object with the new MACs, and finally update the $this->interfaces array.
Your custom mapper must return the SnmpResult object for this data to make it back to Sonar. So, back to our map function — we need to call this new private method, update the SnmpResult object, and then return it.
As you can see, I added the call to getAttachedCustomers inside the main map function, and I also added another line to the getAttachedCustomers function to push my updated interface array into the SnmpResult object.
You are, of course, free to do anything here — it’s just imperative that you allow the base mapper to run its map function, and that you return an updated SnmpResult object. Take a look at the Ws6Mini class for an example of collecting data here using SSH, or the MikroTik class for an example of collecting data using an API.
Testing
Now it’s time to test this! You could, of course, upload all this data to your poller and see if it works, but there’s a quicker test. The poller has a PHP shell built in, powered by PsySH. You can access it by typing vendor/bin/psysh from the root poller directory.
From in here, you can instantiate your new mapper and test it in real time. Let’s step through it! First, we need to instantiate a Device object. We’re going to fake out some of the data in it that we don’t care about to test this. One of the items required is a MonitoringTemplate object. Below, I’ve added a fake array of data you can paste in to help create a default one. Make sure to replace the snmp_community and snmp_version values with appropriate information if you are using SNMP inside your customer mapper — this is where that information will come from.
$templateJson = [
'icmp' => false,
'snmp_version' => 2,
'snmp_community' => 'your snmp community',
'snmp3_sec_level' => null,
'snmp3_auth_protocol' => null,
'snmp3_auth_passphrase' => null,
'snmp3_priv_protocol' => null,
'snmp3_priv_passphrase' => null,
'snmp3_context_name' => null,
'snmp3_context_engine_id' => null,
'oids' => [],
];Now we can create a monitoring template.
>>> $template = new Poller\Models\MonitoringTemplate((object)$templateJson);
=> Poller\Models\MonitoringTemplateNow we can instantiate our Device object. Make sure you enter a valid IP address for the device so we can test the mapper on it.
>>> $device = new Poller\Models\Device(1, (object)[‘ip’ => ‘192.168.100.1’, ‘type’ => ‘network_sites’, ‘polling_priority’ => 1, ‘snmp_overrides’ => []], $template);
=> Poller\Models\DeviceWhen instantiating the Device class, the first input is an inventory item ID (which doesn’t matter for testing), the second is configuration data (of which, you only need to worry about updating ip), and the third is the monitoring template.
Now we can test out the device mapper! All the device mapper requires is the Device object in its constructor, so let’s do that. For my test, I’m using the CanopyPMPAccessPoint mapper, but you would instantiate your new class here.
>>> $mapper = new Poller\DeviceMappers\Cambium\CanopyPMPAccessPoint($device);
=> Poller\DeviceMappers\Cambium\CanopyPMPAccessPointAlmost there! Now we just need an SnmpResult object to pass into the map function. To instantiate that, we need an SnmpResponse object, but we can just create an empty one.
>>> $snmpResult = new Poller\Models\SnmpResult(new Poller\Models\SnmpResponse([]), 1);
=> Poller\Models\SnmpResultFinally, we can call map on our mapper, get the updated SnmpResult back and see what it looks like.
>>> $snmpResult = $mapper->map($snmpResult);
=> Poller\Models\SnmpResultCheck out the SnmpResult class for examples of what you can see here. A quick and dirty way to see everything is to just call ->toArray() on it.
While there’s some effort to run through here to do the initial setup, it’s easy and copy and paste in, and it gives you an immediate view at your mapper response, and to see if there are any errors or exceptions thrown.
Opening a Pull Request
Once you have a working mapper you’re happy with, push your changes back up into your fork, then navigate back to the poller repository on GitHub. Click the Pull Requests tab at the top, and open a new pull request with your changes. We’ll review them and either offer feedback or accept your new mapper to be made available in the public repository to everyone. Thanks!
