High-speed Solutions

The WildPackets Network Analysis and Monitoring Blog covers enterprise networking news from recent standards, such as 802.11n, and upcoming technologies, such as 100G, to pressing everyday issues around wireless, VoIP, security, and network performance management.

Savvius 2017 Security Predictions

With 2017 right around the corner, we have a few predictions for what may take place in the security domain next year.

Staying ahead of the curve

As we mentioned in our networking predictions blog post, 2017 will be a year in which solution vendors come under increasing pressure to integrate products into standardized work flows while providing unique value-add features that address cyber threats. At Savvius we achieve this balance by adhering to commonly accepted industry standards and by not trying to reinvent the wheel where technologies are readily available in the market. At the same time, we are able to complement the industry by applying our expertise in packet analytics and automated data collection to provide industry-leading troubleshooting and security capabilities that result in rapid resolution of network and security investigations.

Unlike the networking industry, which is already quite mature and facing considerable pressure to consolidate, the security space is still a veritable Wild West of competing technologies and approaches. One thing is certain. 2017 breaches resulting in the exfiltration of sensitive data will continue to rise. As an active participant in the security industry, Savvius will continue to play its role helping companies prepare for a breach and minimize its impact.

2017 Security Predictions

  1. Security teams will turn to network engineers for the truth contained in packet data as metadata and log data are increasingly compromised. At the same time, security analytics based on network data will become the “hot” topic for presentations at security conventions.
  1. Sophisticated, state-sponsored security breaches will continue to increase. These adversaries are becoming more adept at bypassing traditional security measures, so as the number of breaches rises, network engineers will increasingly find themselves being called upon to help security investigations. They will need to provide critical network packet data that efficiently answers the who, what, when and how of the intrusion – even weeks or months after being discovered.
  1. Security stack complexity will continue to increase even more rapidly than attack surfaces, greatly increasing the tension between doing business (having low-friction systems and processes) and being in business (avoiding major security incidents), making it vital that enterprises have the capability to conduct rapid, accurate investigations into security incidents.
  1. Security teams will be increasingly inundated by incidents requiring investigation. The only solution is to automate the routine parts of their workflow to help speed up the analysis process. Smart hackers find ways to disguise attacks as low-priority issues making quantity of investigations as important as quality. Automating data collection and alert correlation techniques will help these teams analyze alerts as they come in so that low-level alerts don’t fly under the radar and go unchecked. With adequate automation technology in place, security analysts can expect up to a five-fold increase in the number of alerts that can be checked by the same manpower.
  1. Security teams will see their budgets increase, but demands on their time and expertise will increase even more. The choice is between tolerating increased risk or increasing the efficiency of the security team through automation and machine intelligence.

Check out our 2017 predictions for the network space here.


Integrating Savvius Vigil With New Event Sources

The topic of this blog entry is how to add support for new IDS/IPS and SIEMs to Savvius Vigil. First of all, let’s review what Savvius Vigil is. Savvius Vigil is a unique security appliance that was specifically designed to capture just the security related packets for longer periods of time, so when a breach happens, the bread crumbs are still there to aid the security investigation in discovering how the breach occurred, and what effect it had on the rest of the network. To achieve this amazing feat, Vigil integrates with IDS/IPS and SIEM vendors.

Out of the box, Vigil has support for many of the most popular IDS/IPS and SIEMs, and support for new ones are added in each release. Although the events from most IDS/IPS and SIEMs are pretty similar, they are all different, and can and often do change with new releases. There are also new IDS/IPS and SIEMs coming out all the time. Because of this constantly changing world, it is important to be able to add support in Savvius Vigil for changing and new event formats from different IDS/IPS and SIEMs.

To parse the events Vigil uses Logstash, which is the L in the increasingly popular ELK stack. Logstash is open source, and is commonly used in the log collection world with ELK, where it parses events from logs, and forwards them to the elasticsearch database. In the Savvius Vigil security appliance, Logstash is used as a middle man to parse the events from IDS/IPS and SIEMs and forward them onto Vigil, which uses the events to decide which packets to write to disk. Savvius Vigil has a 128G RAM ring buffer, so it can save packets from 5 minutes ago. This is important in order to capture packets of a flow related to a security event that is triggered by a packet somewhere in the flow. Packets captured by Savvius Vigil that are not associated with any event are not captured to disk.

In a previous blog entry, we propose the idea of adding elasticsearch and kibana to Savvius Vigil, and the many advantages of using it as the packet capture appliance, as well as the SIEM:
https://blog.savvius.com/savvius-elk-best-siem-ever/. In this blog entry we are focusing on the Logstash piece to parse the events. Whether the events are used to save the events for use in the Vigil SIEM, or to capture the packets of the flow, the same Logstash parser can be used.

Since Vigil ships with Logstash, which at Savvius we refer to as the Vigil Universal Parser, we do not have to install it. Vigil also ships with a couple of Logstash configuration files for different IDS/IPS and SIEMs which can be used as templates to write new parsers. More good news is that many Logstash parsers for IDS/IPS and SIEMs have already been written for use with ELK, which can be used as templates for Vigil. A collection of these can be found here on github: https://github.com/SMAPPER/Logstash-Configs/tree/master/configfiles.

For this exercise, we are going to do something a little different. Instead of adding support for events for a new IDS/IPS or SIEM, we are going to add support for Vigil’s own expert events. Vigil Expert Events are similar to most IDS/IPS events in that they include the source and destination IP address of the packet or flow that triggered the event. Vigil expert events are also different because they are not limited to security events, but include network performance events as well. A few points to make about this.

1)    Network performance events can be considered security related events because a change in the performance characteristics of your network can be the result of a security related issue.
2)    If you are storing packets in order to troubleshoot network performance related problems, it seems logical that you would want to store the packets related to a network performance expert event.
3)    Omnipliance Expert Events are not just limited to the analysis of a single packet, but can maintain state between packets. For example, there is an Expert that tracks ESP Out of Sequence packets on a per flow basis. Thirdly, Omnipliance expert events are extensible through a Plugin API. One very good example of this is the filter script plugin. You can learn more about here: https://mypeek.savvius.com/view_solution.php?id=126

Alright. Let’s get our hands dirty, and configure Vigil to generate events.

Omniengine Syslog Configuration

First we need to run Omnipeek and connect to the Vigil appliance. To allow Omnipeek to connect to a Vigil appliance, run regedit, navigate to the following key, and change the ConnectToVigil field value to 1.



Run Omnipeek, and go to Settings->Notifications. Next, select Insert to create a new notification entry and configure it for Syslog. For our purposes, we only want the expert events to be saved, so disable all the source entries except for Expert. In the Recipient field, enter localhost. Hit OK, and enable the new notification for all of the 4 severities.



Keep in mind that this configuration assumes we are running the analysis capture and vigil capture on the same machine. It is the analysis capture that is going to generate the events that tells the Vigil Capture which packets to save to disk.

Omniengine Analysis Capture

To create a new analysis capture in Omnipeek, go to the captures tab and create a new capture. Select the same interface as the Vigil Capture, go to the Analysis Options tab, and only enable the Expert Analysis.


You can see in the above screenshot that we now have two captures, one for Vigil to save packets, and one for analysis to figure out which packets to save.

Expert Event Tuning

Once you have created the capture, you can also go to the Expert Event Finder and fine tune the expert events. This is highly recommended, since the default settings for the Expert Events can be very noisy. The idea here is that you want to capture the packets that really matter, so you can keep them for longer periods of time.


Configuring Logstash

Now let’s SSH into the Vigil appliance and add a new Logstash parser. Cd to /etc/logstash/conf.d, and create a new file with “vi omni2vigil.conf”. In this file, copy the following text, and save the file.

input {

file {

path => “/var/log/syslog”

type => events

start_position => “beginning”

tags => [“phase1”]


file {

path => “/var/log/omni2vigil.log”

start_position => “beginning”

tags => [“phase2”]




filter {

if “phase1” in [tags] {

grok {

match => { “message” => “^%{SYSLOGTIMESTAMP:logtimestamp}\s%{DATA:source_type}\sOmni:\sExpert:\s%{GREEDYDATA:alert},%{GREEDYDATA}Packet%{GREEDYDATA}\(%{IPV4:SrcIp}:%{INT:SrcPort} -> %{IPV4:DstIp}:%{INT:DstPort}\)” }

match => { “message” => “^%{SYSLOGTIMESTAMP:logtimestamp}\s%{DATA:source_type}\sOmni:\sExpert:\s%{GREEDYDATA:alert},%{GREEDYDATA}Packet%{GREEDYDATA}\(%{IPV4:SrcIp} -> %{IPV4:DstIp}\)” }


date {

locale => “en”

match => [“logtimestamp”, “MMM dd HH:mm:ss”, “MMM  d HH:mm:ss”]

target => “@timestamp”



if “phase2” in [tags] {

grok {

match => { “message” => “%{GREEDYDATA:other}” }


mutate { gsub => [‘other’, “‘”, ‘”‘] }




output {

if “phase1” in [tags] {

elasticsearch {

action => “index”

hosts => [“localhost:9200”]

index => “logstash-%{+YYY.MM.dd}”


file {

path => “/var/log/omni2vigil.log”

message_format => “%{logtimestamp} [–] vigil-common {‘source_type’: ‘%{source_type}’, ‘time_local’: ‘%{logtimestamp}’,’time_utc’: ”,’sig_gid’: 0,’sig_sid’: 0,’sig_rev’: 0,’sig_str’: ‘%{alert}’,’classification’: ‘Expert’,’severity’: 1,’protocol’: ‘TCP’,’ip_src_combo’: ”,’ip_src_address’: ‘%{SrcIp}’,’ip_src_port’: %{SrcPort},’ip_dst_combo’: ”,’ip_dst_address’: ‘%{DstIp}’,’ip_dst_port’: %{DstPort},’source_type’: ‘none’, ‘ip_ids_combo’: ”,’ip_ids_address’: ‘%{host}’,’ip_ids_port’: 0}”



if “phase2” in [tags] {

file {

path => “/var/log/auth.log”

message_format => “%{other}”




This parser is going to monitor the syslog file for events that we configured the Omniengine to send there, reformat them into the Vigil common format, and write them to /var/log/auth.log.   Since our final output needs to have double quoted fields, but Logstash does not like double quoted fields inside a double quoted string, we have a 2 stage process in order to convert the single quotes into double quotes. There are 3 files involved, that perform the following functions:

1)    Syslog: Pick up the events from the Omniengine in the syslog file. In the filter for stage 1, two matches are used, one for TCP with ports, and one for ICMP without ports. The relevant fields are parsed out of the syslog event, and formatted into the vigil-common event format.
2)    Omni2vigil.log: At the end of stage 1, the vigil-common formatted events are written to omni2vigil.log.  At the beginning of stage 2 the formatted events are read from omni2vigil.log, and single quotes are converted to double quotes.
3)    Auth.log: At the end of stage 2, the double quoted vigil-common event is written to auth.log, where the Vigil appliance will be waiting for it.

If any anyone comes up with a better way to do this, please let me know.

Running Logstash

Now that we have a Logstash configuration file, let’s run it with the following command:

/opt/logstash/bin/logstash -f /etc/logstash/conf.d/omni2vigil.conf

This command should be set up as a script, and run at boot. With an analysis capture running in the engine, and this logstash command, you should be seeing events in the /var/log/auth.log that look like this:

Nov  2 08:39:37 [–] vigil-common {“source_type”: “vigil”, “time_local”: “Nov  2 08:39:37″,”time_utc”: “”,”sig_gid”:
0,”sig_sid”: 0,”sig_rev”: 0,”sig_str”: “TCP Fast Retransmission (by time) (0.000 seconds from packet
53454607)”,”classification”: “Expert”,”severity”: 1,”protocol”: “TCP”,”ip_src_combo”: “”,”ip_src_address”:
“”,”ip_src_port”: 46168,”ip_dst_combo”: “”,”ip_dst_address”: “”,”ip_dst_port”: 443,”source_type”: “none”, “ip_ids_combo”: “”,”ip_ids_address”: “”,”ip_ids_port”: 0}

The above event is known as a vigil-common event, and is in json format. The vigil-common event type was introduced in Vigil 2.0 to allow for new event inputs into Vigil, like the one we are doing here.  You may notice that some of the fields in the vigil-common event are empty. Some of the fields also have default values. This is because Vigil is designed to receive event from IDS/IPS and SIEMs, whose events are security specific, and although are formatted differently, have common fields. The Omniengine expert events do not have all of these fields, so we provided default values for some of them.

Vigil Client

Finally, we will use the Vigil Client to configure Vigil to accept these new events. Since earlier we were connected to Vigil with Omnipeek using the vigiladmin user, we will need to disconnect Omnipeek from Vigil before connecting to Vigil with the Vigil Client. This is because only one client at a time can be connected to Vigil. So disconnect from Omnipeek, and connect to Vigil using the Vigil Client, and go to the Admin tab.


Hit the Add button in the Alert Sources section, and configure it as shown below.


Hit OK, and we should be done with configuration. Take a break, you deserve a cup of tea. Another reason to take a break, is that it will take between 5-20 minutes before events from Omniengine to appear in the Vigil client. This is because of the way the Vigil packet buffer works that allows Vigil to store packets related to the event up to 5 minutes before the event occurred.  When you return, it will be time for the finale. In the Vigil Client dashboard, you should see the Omniengine listed in the Alert Sources panel, and Omniengine events listed in the Active Alerts and Latest Completed Alerts panels.


At this point you can go to the Alerts tab, pick a range of time, select an alert that you want packets for, and hit the Create Packet File button. This will create a forensic search on the Vigil Appliance, and download the packets related to that event. Once the packet file is downloaded, you can open it with Omnipeek.


And there you have it, a DIY on how to configure Vigil to use its own expert events to determine which packets to capture. I hope you enjoyed this write-up, and will stay tuned for more adventurous blog entries on how to implement interesting and useful integrated solutions with Savvius software and hardware products. And by the way, if there is a topic you would like us to write more about, please let us know. You can contact us at info@savvius.com.

To learn more visit our events page and sign up for a webinar!

Written by: Chris Bloom, Technology Evangelist at Savvius


Contact Us Savvius Blog Follow Savvius on Twitter Like Savvius on Facebook Follow Savvius on LinkedIn Follow Savvius on YouTube Follow Savvius on Slideshare