Devstack is the most commonly used project for OpenStack development. Wouldn’t it be cool to have a supporting software which analyzes the network infrastructure and helps us troubleshoot and monitor the SDN solution that Devstack is deploying?
Skydive is an open source project aimed to help troubleshooting SDN systems by analyzing the data that it collects from the network infrastructure. It is written to be SDN agnostic.
This post explains in detail about how we can plug Skydive into Devstack.
Skydive components
Skydive needs two of its components to be running for it to work.
These services are run by the Skydive’s plugin to Devstack. You should add the following to your local.conf to initialize and configure the Skydive plugin.
enable_plugin skydive https://github.com/skydive-project/skydive.git
1. Skydive agents
These are data collecting agent. Agent will typically run alongside other network agents on a compute node.
To enable Skydive agent in Devstack, the following lines should be added to the local.conf.
enable_service=skydive-agent
2. Skydive analyzers
This is the information submission and retrieval component. It can also leverage information from SDN controllers. This component would be run typically in controller nodes.
To enable skydive analyzer in devstack, following should be added to the local.conf.
enable_service=skydive-analyzer
Plugin configuration parameters
In addition to the above inclusion, following parameters can be added in your Devstack’s local.conf.
- SKYDIVE_ANALYZER_LISTEN
Address on which skydive analyzer process listens for connections. Must be in ip:port format.
Default:
$SERVICE_HOST:8082 - SKYDIVE_AGENT_ANALYZERS
Inform the agent about the address on which analyzers are listening. Must be in ip:port format.
Default:
$SKYDIVE_ANALYZER_LISTEN - SKYDIVE_AGENT_LISTEN
ip:port address on which skydive agent listens for connections.
Default:
127.0.0.1:8081 - SKYDIVE_AGENT_ETCD
Configure the skydive agent with the etcd server address. Skydive analyzer embeds its own etcd. Hence the IP address in this url should be that of the analyzer.
Default:
http://$SERVICE_HOST:2379 - SKYDIVE_CONFIG_FILE
The path for the generated skydive configuration file.
Default:
/tmp/skydive.yaml - SKYDIVE_AGENT_PROBES
List of agent probes to be used by the agent.
Default:
netlink netns ovsdb neutron - SKYDIVE_OVSDB_REMOTE_PORT
Remote tcp port for ovsdb server.
Default:
6640 - SKYDIVE_LOGLEVEL
Set the default log level for Skydive components.
Default:
INFO
Sample two node deployment
Inside the Devstack folder of the Skydive sources there are two local.conf files.
These files can be used to deploy two Devstacks with Skydive. The first file will install devstack with both Skydive analyzer and agent. The second file will install devstack with only the Skydive agent.
With the above conf files, Skydive will be set with the probes for OpenvSwitch and Neutron. It will be set to use Keystone as authentication mechanism, so the credentials will be the same as that of the admin.
Once you have your Devstacks deployed, the WebUI on the Analyzer should show something similar to the following capture.
http://$SERVICE_HOST:8082
Capture traffic
Now we have our two nodes up and running we may want to start capturing packets. The following command can be used in order to start a capture on all the br-int bridges.
Before starting to capture, Skydive needs the credentials exported. Since the configuration is such that Skydive uses keystone for authentication, you can export the following.
export SKYDIVE_USERNAME=admin
export SKYDIVE_PASSWORD=password
To start a capture:
$ skydive client -c /tmp/skydive.yaml capture create \
--probepath "*/br-int[Type=ovsbridge]"
$ skydive client -c /tmp/skydive.yaml capture list
{
"*/br-int[Type=ovsbridge]": {
"ProbePath": "*/br-int[Type=ovsbridge]"
}
}
You can verify your capture by querying the elasticsearch after a datapath action.
curl -s http://$SERVICE_HOST:9200/_search | jq .hits.hits[0]
Conclusion
With the kind of data that skydive collects and presents, it can come in handy when debugging SDN problems while developing. Thanks to Devstack’s plugin infrastructure, adding Skydive to devstack is really easy.