Introduction

Welcome to the CVE checker user guide. The purpose of this document is to guide you through the installation and usage of CVE checker. But before we start with that, a little warning is in place...

CVE checker does not provide security for your system. It is nothing more than a tool that attempts to identify installed software on your system and match those against the online database of CVE entries. As the identification of installed software depends on simple rules that are added by the community, it is very likely that it only detects a small portion of the installed software on your system (especially if you are running software that isn't all that popular). And even if the tool detects the software appropriately, it still requires that the CVE entry mentions this software with the same vendor and product name as this tool detects - if not, CVE checker will not be able to match the installed software against the online CVE database.

Okay, now that that is out of the way, let's start with the CVE checker installation.

Installation

Requirements

The cvechecker tool requires the following packages to be installed:

Installation

If your distribution already offers cvechecker through its package management system, I strongly advise that you use that. The installation instructions in this section assume that you do the installation manually (and should reflect the steps that the distribution package management system does for you).

First, download the latest cvechecker package from https://github.com/sjvermeu/cvechecker/wiki/Download (or https://github.com/sjvermeu/cvechecker/releases), then extract it into a temporary directory:

~$ tar xvzf cvechecker-1.O.tar.gz

You need to select the database you want to use. cvechecker currently supports mysql and sqlite, and this support is enabled through the ./configure --enable-sqlite3 or ./configure --enable-mysql options. You can select both too (they are not mutually exclusive) but you need to select at least one of them.

Inside the extracted folder, first run ./configure with the proper --enable-<database> options, followed by make and make install. If you use the default settings, it will install the files inside /usr/local, so you might need root privileges to perform the make install step.

~$ cd cvechecker-1.0
~$ ./configure --enable-sqlite3 
...
~$ make
...
~$ sudo -i
~# make install

If the ./configure step fails (especially when using the live code rather than a specific release), first run the following command:

~$ autoreconf --force --install

Finally, create a group called cvechecker and change the permissions of the installed folders so that this group has write privileges inside the directory. These steps are performed when executing make postinstall (you almost definitely need root privileges now):

~# make postinstall

Initial Configuration

With the software now installed, you need to perform an initial configuration. This includes the setup of the databases, creation of the user you are going to use to run cvechecker and initial pull of the CVE entries from the Internet.

First, create a user that you will use to run cvechecker. The tool does not require root privileges, so I recommend not to use root. I personally use a dedicated user whose sole purpose is to, on a daily basis, pull the latest updates from the Internet, generate a report and mail that report to me. Make sure that this user is inside the cvechecker group:

~# gpasswd -a youruser cvechecker

Log on as that user, make sure his primary group is cvechecker and set the umask to 002 or higher (such as 007).

~# su youruser -
~$ newgrp cvechecker
~$ umask 007

Now, initialize the databases by running cvechecker -i. This will create the sqlite3 databases or, if MySQL is selected, create the appropriate tables and indexes in the selected MySQL database (the tool does not create a MySQL database itself).

~$ cvechecker -i

Configuration File

The cvechecker command uses a configuration file which it looks for in the location pointed to by the CVECHECKER_CONFFILE variable or, if that variable is not set, the users' home directory (${HOME}/.cvechecker.rc), /usr/local/etc/cvechecker.conf or /etc/cvechecker.conf, in that order. The first configuration file that parses correctly is used, the rest is ignored (so there is no accumulation of configurations). Below, you'll find an example configuration file which shows both the sqlite3 and mysql support:

#
# Generic settings
#
dbtype = "mysql";
#dbtype = "sqlite3";
cvecache = "/var/lib/cvechecker/cache";
datadir = "/usr/share/cvechecker";
stringcmd = "/usr/bin/strings -n 3 '@file@'";
version_url = "https://raw.github.com/sjvermeu/cvechecker/master/versions.dat";
#userkey = "servertag";

#
# For SQLite3
#
sqlite3: {
  localdb = "/var/lib/cvechecker/local";
  globaldb = "/var/lib/cvechecker/global.db";
};

#
# For MySQL
#
mysql: {
  dbname = "cvechecker";
  dbuser = "cvechecker_rw";
  dbpass = "passwordforcvechecker_rw";
  dbhost = "mysql.company.com";
};

Most of the parameters (like cvecache, datadir, version_url and even localdb and globaldb) will most likely be set correctly already during the installation. However, it doesn't hurt to validate if the locations mentioned are correct and that the user that will be used to run cvechecker has the rights to write (to) files in the locations.

Initialize SQLite3 Database

If you opt to use the sqlite3 databases, you need to initialize those by running cvechecker -i. Because cvechecker uses a lot of data, it will create various database files to spread the data due to performance reasons. These locations are mentioned in the configuration file.

~# cvechecker -i

This is all that is needed to initialize the SQLite3 databases.

Initialize MySQL Database

If you opt to use the MySQL database, you first need to create the database in your MySQL server. Installing and configuring a MySQL server is beyond the scope of this document. Once a database is created (say "cvechecker"), you can either initialize it through cvechecker -i (if the user you connect to the database to has sufficient privileges to drop and create tables and indexes) or use the dump file mysql_cvechecker.sql available in /usr/share/cvechecker/.

~# cvechecker -i

It is strongly advised that the user you use has limited rights. Only the initialization requires additional rights. When you are using cvechecker, it is recommended that two roles are used:

We will talk about these roles later in this document.

Load CVE and version matching rules

Our last step is to load the upstream data in the databases.

The upstream data consists out of

These steps are automated in a tool called pullcves. This will take a very long time, so please be patient (loading over half a million CVE entries in a database is a time consuming - but one-time - activity). Future pulls will not take this much time as they will not redownload the CVE entries from all previous years (unless you ask it to).

~$ pullcves pull

Upgrades

cvechecker tries to detect if its back-end was created with an earlier version of cvechecker and update the back-end accordingly. For sqlite3 databases, this usually isn't a problem as the tool has the rights to add tables and indexes when needed.

However, when using MySQL databases, it is very likely that the user you use is not authorized to update the back-end. To fix this, first run cvechecker on the administrative host (where a more privileged MySQL account can be used). It will update the back-ends for all users as well. When new(er) versions are made available, we will also add the necessary upgrade SQL scripts to launch so that you do not have to run cvechecker with a database owner account.

Regular Runs

Whenever you want to perform a regular cvechecker run, you should schedule five steps. If you use sqlite as the database backend, all steps are to be scheduled on the same system. If you use MySQL however, it is recommended to schedule the administrative tasks on a single system (the "master" that is responsible for keeping the upstream data up-to-date) and the operational tasks on all systems where you want to run cvechecker on (and gather reports from).

The administrative tasks are:

  1. pull in the latest CVE entries as well as software/version detection rules

The operational tasks are:

  1. generate a list of files to scan

  2. gather installed software/version information

  3. output which CVE entries might affect your system

  4. generate a report informing you about the CVE entries

All these steps should be performed by the user you use to run cvechecker, so don't forget:

~$ newgrp cvechecker
~$ umask 007

Pull Data

To pull the necessary data from the Internet, run pullcves pull:

~$ pullcves pull

Generate List of Files

cvechecker uses a text file which sums up the files to scan. I recommend that you provide it with a list of all executables files plus /proc/version (on Linux). The latter allows cvechecker to verify if there are kernel-related CVE entries for your Linux kernel.

~$ find / -type f -perm -o+x > scanlist.txt
~$ echo "/proc/version" >> scanlist.txt

It is strongly advised to either umount or exclude file systems on which no useful files are located (or that belong to different operating systems, like a mounted Windows partition). You can use the -mount (or -xdev) option of find to stay within the selected device (partition), or exclude directories (and their subdirectories), like so:

~$ find / -path /mnt -prune -o -path /media -prune -o -type f -perm -o+x > scanlist.txt

Gather List of Installed Software/Versions

Import the generated scanlist.txt file in cvechecker. During the import, cvechecker will output the software/versions it detected:

~$ cvechecker -b scanlist.txt

Integrating with a package manager

You can integrate cvechecker with your package manager, telling cvechecker to only add (or remove) files part of an installed (or uninstalled) package rather than having to feed a full file system listing. To do so, tell your package manager to call cvechecker -d -b <file> after each succesfull installation, or cvechecker -D -b <file> before a desinstallation. The -d option tells cvechecker to add the file content to the database (rather than replace) whereas -D tells cvechecker to delete those entries in the database that are also in the file.

A simple example for Gentoo would be to create a script (/etc/portage/bashrc) which calls other scripts based on the ebuild phase, such as /etc/portage/bashrc.d/package.postinst or /etc/portage/bashrc.d/package.prerm. These scripts can then call qlist to create a file containing the package file content which is then passed on to cvechecker.

~$ cat /etc/portage/bashrc
if [ -x "/etc/portage/bashrc.d/package.${EBUILD_PHASE}" ];
then
  source "/etc/portage/bashrc.d/package.${EBUILD_PHASE}"
fi

~$ cat /etc/portage/bashrc.d/package.prerm
TMPFILE=$(mktemp);
[ ! -f "${TMPFILE}" ] && die "Cannot create temporary file.";
qlist ${CATEGORY}/${P} > "${TMPFILE}";
cvechecker -D -b "${TMPFILE}";
rm "${TMPFILE}";

Adding through standard input

The cvechecker command will also accept standard input for the --binlist (-b) option when the minus sign is used:

~$ find /usr/bin -type f | cvechecker -b -
Searching for known software titles...
 - Found match for /proc/version:    cpe:/o:linux:linux_kernel:3.10.5-hardened:::

Output Matching CVE Entries

Make cvechecker output the matching CVE entries. A standard output might look like this:

~$ cvechecker -r
File "/usr/bin/kbxutil" (CPE = cpe:/a:gnupg:gnupg:2.0.15:::) on host localhost (key localhost)
  Potential vulnerability found (CVE-2010-2547)
  Full vulnerability match (incl. edition/language)
File "/usr/bin/gpgsm" (CPE = cpe:/a:gnupg:gnupg:2.0.15:::) on host localhost (key localhost)
  Potential vulnerability found (CVE-2010-2547)
  Full vulnerability match (incl. edition/language)
File "/usr/bin/watchgnupg" (CPE = cpe:/a:gnupg:gnupg:2.0.15:::) on host localhost (key localhost)
  Potential vulnerability found (CVE-2010-2547)
  Full vulnerability match (incl. edition/language)

However, if you would like to parse the output so that you can generate a better-looking report, ask cvechecker to use CSV output:

~$ cvechecker -r -C
Outputversion,File,CPE,CVE,Matchtype,Hostname,Userkey
2,/usr/bin/kbxutil,cpe:/a:gnupg:gnupg:2.0.15:::,CVE-2010-2547,1,localhost,localhost
2,/usr/bin/gpgsm,cpe:/a:gnupg:gnupg:2.0.15:::,CVE-2010-2547,1,localhost,localhost
2,/usr/bin/watchgnupg,cpe:/a:gnupg:gnupg:2.0.15:::,CVE-2010-2547,1,localhost,localhost

The default behavior of cvechecker is to only report on those CVEs who specifically target the software and version you have installed. However, you might want to be informed about software matches for CVEs that affect higher versions than the versions installed on your system. Not all CVEs contain a full list of all versions affected but rather mention only the highest version vulnerable. The default reporting within cvechecker wouldn't be triggered when this is the case, but if you add -H or --reporthigher, cvechecker will also report on higher versions.

(Optional) Generate A Report

You can use the cvechecker output to generate a report. Within the cvechecker distribution, sample reporting files are available that show you a simple method. Of course, reporting is not part of this tools' goals; the primary purpose of this step is to show you that the output can indeed be used for reporting purposes. To facilitate (simple) reporting, a cvereport script has been provided which uses the sample reports currently. In the future, this will be enhanced with distribution-specific checks.

~$ cvereport -d /home/john/cvereport-20100902/ /home/john/cvechecker/acknowledgements.xml

The default report will use the default cvechecker report. If you want to report using higher version matches as well, use -D instead of -d.

Below is a sample acknowledgements XML file. Its use should be self-explanatory:

<?xml version="1.0" encoding="UTF-8"?>
<acknowledgements>
  <!-- I chose to use the CVE number as id for the resolution/comment text, but this is not mandatory -->
  <resolution id="CVE-2008-4101">Has been solved by security update #17 (vim-1.7-b3)</resolution>
  <comment id="CVE-2010-2547">Under investigation, bug opened on distributions bug site (#332329)</comment>
  <!-- The @comment or @resolution should refer to one of the above descriptions. 
         Its use however is optional (you don't need to give comments) -->
  <file name="/usr/bin/kbxutil" cve="CVE-2010-2547" state="acknowledged" comment="CVE-2010-2547" />
  <file name="/usr/bin/vim" cve="CVE-2008-4101" state="irrelevant" resolution="CVE-2008-4101" />
</acknowledgements>

The final document (report.html) can then be mailed, published or in any other way sent to you.

If the report shows new CVE entries matching your system, investigate those CVE entries by first acknowledging and, if necessary, either update the software on your system or mark the entry as irrelevant.

Detected Software

The tool also supports listing the detected software/version on your system. This allows you to verify if the tool can detect the software you want to track CVE reports of.

~$ cvechecker -s
Detected vendor="wpa_supplicant", product="wpa_supplicant", version="0.6.10", update="", edition="", language="" on host="localhost", userkey="localhost"
...

You can also request the list of files that were used by cvechecker to identify the software:

~$ cvechecker -S
Detected vendor="wpa_supplicant", product="wpa_supplicant", version="0.6.10", update="", edition="", language="" on host="localhost", userkey="localhost"
Files that contributed to this detection:
  - /usr/sbin/wpa_supplicant
  - /usr/bin/wpa_cli

These options also support the -C switch (to enable CSV-like output for easier parsing);

~$ cvechecker -S -C
Outputversion,Vendor,Product,Version,Update,Edition,Language,Hostname,Userkey,Files
2,wpa_supplicant,wpa_supplicant,0.6.10,,,,localhost,localhost,/usr/sbin/wpa_supplicant /usr/bin/wpa_cli

Watch Lists

With cvechecker, you can also "pretend" to have particular software (or hardware or operating systems) installed on your system. By feeding this so-called watchlist to cvechecker, it will add these titles to the database and assume they are also installed on the system.

An example watchlist is given next. Right now, the cvechecker application requires at least 8 fields so a number of trailing field separators might be needed. The example shows a watchlist for two applications:

~$ cat watchlist.txt
cpe:/a:dropbear_ssh_project:dropbear_ssh:2016.74:::
cpe:/a:libexpat:expat:2.2.0:::

It is recommended to use the watch lists simultaneously with the binary file listing. If you use it separately, don't forget to use the delta options. Otherwise, all detected software versions will be cleared from the database.

~$ cvechecker -b binlist.txt -w watchlist.txt
~$ cvechecker -w watchlist.txt --deltaonly

Configuring cvechecker

The cvechecker tool uses a configuration file to find out where to store its intermediate and final data (such as downloaded XML files, CSV files, database files).

The tool first looks for the configuration file pointed towards by the CVECHECKER_CONFFILE variable. If that environment variable doesn't exist, then .cvechecker.rc inside the users' home directory is looked for. If it can't find it, it will look for /usr/local/etc/cvechecker.conf and, if that file doesn't exist either, /etc/cvechecker.conf.

Configuration File Format

The cvechecker configuration file currently requires 6 configuration items:

#
# Generic settings
#
dbtype = "mysql";
#dbtype = "sqlite3";
cvecache = "/var/lib/cvechecker/cache";
datadir = "/usr/share/cvechecker";
stringcmd = "/usr/bin/strings -n 3 '@file@'";
version_url = "https://raw.github.com/sjvermeu/cvechecker/master/versions.dat";
#userkey = "servertag";

#
# For SQLite3
#
sqlite3: {
  localdb = "/var/lib/cvechecker/local";
  globaldb = "/var/lib/cvechecker/global.db";
}

#
# For MySQL
#
mysql: {
  dbname = "cvechecker";
  dbuser = "cvechecker_rw";
  dbpass = "passwordforcvechecker_rw";
  dbhost = "mysql.company.com";
};

These items describe the following resources:

Warning

If you need to change parameters, make sure that the file containing the changed values contains all configuration directives (and not just the changed one). cvechecker does not "fall back" to other configuration files if a parameter is missing.

MySQL Database Usage

When you use cvechecker with MySQL, then the following sections might be of use to you...

User Roles

When using MySQL database(s), it is recommended to use two separate roles:

A third role "could" be created (which allows cvechecker to (re)initialize its databases using cvechecker -i) but this should only be activated during the (re)initialization and locked again when the change is finished.

InnoDB or MyISAM?

The cvechecker tool currently uses InnoDB tables. The main (only) reason for this is that the tool uses foreign key definitions to keep referential integrity within the database. However, untrained administrators might find InnoDB a lot slower to work with than MyISAM. It is perfectly possible to use MyISAM tables instead of InnoDB (cvechecker does not use the cascading delete functionality within foreign key definitions):

  1. Change the mysql_cvechecker.sql file by removing all references to FOREIGN KEY ... statements as well as ENGINE=InnoDB.

  2. Load the (changed) SQL file in the MySQL database

  3. Do not use cvechecker -i (as that runs hardcoded SQL statements which will transition to InnoDB again).

If you want to convert when data is already present, the easiest method in my opinion is to dump the information to a .sql file, change the .sql file and load it back in. However, other methods exist as well - which ones I leave as an excercise in Internet search engines for those interested ;-)

Software Matching Rules

The software matching rules, as identified inside the versions.dat file, is pulled in and imported in the cvechecker database by pullcves.

Syntax

The syntax of the versions.dat file is as follows:

,[filestart],1,[fileexpression],[contentexpression],[ao],[vendor],[product],[version],[update],[edition],[language]

The first comma is also used to inform cvechecker about the used column separator. If by any chance a field requires a comma as content, another separator can be chosen - just start the line with the chosen separator character.

The other fields have the following description:

The parts starting from the [ao] field are also known as the CPE or Common Platform Enumeration. See http://cpe.mitre.org for more information.

Some example lines are:

,gzip,1,gzip,^([0-9]\.[0-9][0-9]*),a,gnu,gzip,\1,,,
,vim,1,vim,VIM - Vi IMproved ([0-9\.]+) ,a,vim,vim,\1,,,
,ncftp,1,ncftp,^.*NcFTP ([0-9\.]+)/([0-9]+) .*,a,ncftp_software,ncftp,\1,\2,,

Validating 'versions.dat'

To validate the matching rules, you can ask cvechecker to tell you if, and what CPE data it detects from a specific file:

~$ cvechecker -f /usr/bin/vim
  - Found match for /urs/bin/vim:    cpe:/a:vim:vim:7.2:::

In the above case, it detected for the /usr/bin/vim file vim (product) from vim (vendor) versino 7.2. It was not able to deduce any specific releases/edition/language (but that is not mandatory anyway).

If you want to add additional rules, make sure that the dat-file you will be importing contains the previous rules as well - cvechecker will drop all previous rules while importing new ones.

~$ cvechecker -l /usr/local/var/cvechecker/cache/versions.dat
Loading in new version data file...

Helping with version rules management

The version matching rules are at the core of the cvechecker tool. Without them, the tool is not able to detect much of anything on your system. To allow you to contribute to the version matching rules, the project also contains a tool called cverules.

The cverules command, which currently only supports Gentoo systems but should be easily portable towards other distributions or even platforms, will attempt to generate version matching rules for cvechecker that are currently not part of the project but which should give additional matches on your system.

For instance, if cvechecker currently doesn't know how to detect OpenLDAP versions, but you have OpenLDAP on your system, then cverules might be able to deduce a (generic put working) expression for the version matching rules which will allow cvechecker to identify OpenLDAP. If you then send the output of the cverules to the cvechecker project, then these additional rules will be added to the version matching rules so that your next pullcves pull will pull in these additional rules.