From 802f0d67ece2fa87b1bb4fe407f54b7dbfe66156 Mon Sep 17 00:00:00 2001 From: hagiwara Date: Fri, 1 Apr 2016 14:52:18 +0900 Subject: [PATCH 1/3] fix-english --- README.md | 69 ++++++++++++++++++++++++++----------------------------- 1 file changed, 32 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index 4b3ae707..a132ee23 100644 --- a/README.md +++ b/README.md @@ -5,8 +5,7 @@ Vulnerability scanner for Linux, agentless, written in golang. -[README in Japanese](https://github.com/future-architect/vuls/blob/master/README.ja.md) -We have a slack team. [Join slack team](http://goo.gl/forms/xm5KFo35tu) +[README in Japanese](https://github.com/future-architect/vuls/blob/master/README.ja.md) [![asciicast](https://asciinema.org/a/3y9zrf950agiko7klg8abvyck.png)](https://asciinema.org/a/3y9zrf950agiko7klg8abvyck) @@ -59,10 +58,7 @@ Vuls is a tool created to solve the problems listed above. It has the following ---- -# Hello Vuls - -Describe how to scan the vulnerabilites that is included on the localhost. -Procedure is as follows. +# step-by-step explanation 1. Launch Amazon Linux 1. Enable to ssh from localhost @@ -76,11 +72,11 @@ Procedure is as follows. ## 1. Launch Amazon Linux -- For the purpose of explanation, using the old AMI (amzn-ami-hvm-2015.09.1.x86_64-gp2 - ami-383c1956) +- We are using the old AMI (amzn-ami-hvm-2015.09.1.x86_64-gp2 - ami-383c1956) for this example - Instance size: t2.medium - - Unable to work on t2.small, micro, nano, Because 2.3GB memory is needed during data fetch from NVD for the first time. - - After the initial data fetch has been completed, It is able to run on t2.nano -- Add the following to the cloud-init, to prevent from automatically update at the first launch. + - For the first time, t2.medium and above is required for the data fetch from NVD + - You can switch to t2.nano after the initial data fetch. +- Add the following to the cloud-init, to avoid auto-update at the first launch. - [Q: How do I disable the automatic installation of critical and important security updates on initial launch?](https://aws.amazon.com/amazon-linux-ami/faqs/?nc1=h_ls) ``` @@ -88,7 +84,7 @@ Procedure is as follows. repo_upgrade: none ``` -## 2. Enable to ssh from localhost +## 2. Generate a ssh-key on local machine Create a keypair then append public key to authorized_keys ```bash @@ -113,7 +109,7 @@ $ wget https://storage.googleapis.com/golang/go1.6.linux-amd64.tar.gz $ sudo tar -C /usr/local -xzf go1.6.linux-amd64.tar.gz $ mkdir $HOME/go ``` -Put these lines into /etc/profile.d/goenv.sh +Add these lines into /etc/profile.d/goenv.sh ```bash export GOROOT=/usr/local/go @@ -148,7 +144,7 @@ $ ls -alh cve.sqlite3 -rw-r--r-- 1 ec2-user ec2-user 7.0M Mar 24 13:20 cve.sqlite3 ``` -Now we has vulnerbility data, So start as server mode again. +Now we successfully collected vulnerbility data, then start as server mode again. ```bash $ go-cve-dictionary server [Mar 24 15:21:55] INFO Opening DB. datafile: /home/ec2-user/cve.sqlite3 @@ -159,7 +155,7 @@ $ go-cve-dictionary server ## 5. Deploy vuls -Launch a new terminal, SSH to the ec2. +Launch a new terminal, SSH to the ec2 instance. go get ``` @@ -181,14 +177,13 @@ user = "ec2-user" keyPath = "/home/ec2-user/.ssh/id_rsa" ``` -## 7. Prepare - +## 7. Initializing Vuls ``` $ vuls prepare ``` -## 8. Scan +## 8. Start Scanning ``` $ vuls scan @@ -223,7 +218,7 @@ Package/CPE java-1.7.0-openjdk-1.7.0.91-2.6.2.2.63.amzn1 -> java-1.7.0-openj ## 9. TUI -Vuls has Terminal-Based User Interface to see the latest scan. +Vuls has Terminal-Based User Interface to display the scan result. ``` $ vuls tui @@ -242,14 +237,14 @@ $ vuls tui - Fetch vulnerbility information from NVD, JVN(Japanese), then insert into SQLite. ## Vuls -- Scan vulnerabilities that exist on servers, to get a list of the CVE ID +- Scan vulnerabilities of the servers and generate a list of the CVE ID - In order to get more information of the detected CVE, send HTTP request to go-cve-dictinary -- Send the report by Slack, Email -- System operator can see the latest report by terminal +- Send a report by Slack, Email +- System operator can view the latest report by terminal ---- -# Usecase +# Use cases ## Scan all servers @@ -278,7 +273,7 @@ web/app server in the same configuration under the load balancer # Usage: Automatic Server Discovery -Discovery subcommand discovers active servers which specifed in CIDR range, then print the template of config file(TOML format) to terminal. +Discovery subcommand discovers active servers specifed in CIDR range, then print the template of config file(TOML format) to terminal. ``` $ vuls discover -help @@ -351,7 +346,7 @@ You can customize your configuration using this template. - hookURL : Incomming webhook's URL - channel : channel name. If you set #{servername} to channel, the report will be sent to #servername channel. - In the following example, the report will be sent to #server1 and #server2. + In the following example, the report will be sent to the #server1 and #server2. Be sure to create these channels before scanning. ``` [slack] @@ -371,7 +366,7 @@ You can customize your configuration using this template. - iconEmoji: emoji - authUser: username of the slack team - - notifyUsers: a list of Slack usernames to send Slack notification. + - notifyUsers: a list of Slack usernames to send Slack notifications. If you set ["@foo", "@bar"] to notifyUsers, @foo @bar will be included in text. So @foo, @bar can receive mobile push notifications on their smartphone. @@ -397,7 +392,7 @@ You can customize your configuration using this template. #keyPath = "/home/username/.ssh/id_rsa" #keyPassword = "password" ``` - Items that are not specified in the server section will be set those items of the defualt section. + Items of the defualt section will be used if not specified. - servers section ``` @@ -414,8 +409,8 @@ You can customize your configuration using this template. # "cpe:/a:rubyonrails:ruby_on_rails:4.2.1", #] ``` - You can overwrite the default value that specified in default section. - Vuls supports multiple SSH authentication method. + You can overwrite the default value specified in default section. + Vuls supports multiple SSH authentication methods. - SSH agent - SSH public key authentication (with password, empty password) - Password authentication @@ -496,7 +491,7 @@ scan: ## example -Run go-cve-dictionary as server mdoe before scanning. +Run go-cve-dictionary as server mode before scanning. ``` $ go-cve-dictionary server ``` @@ -505,17 +500,17 @@ $ go-cve-dictionary server ``` $ vuls scan --report-slack --report-mail --cvss-over=7 ``` -Examples of the above are the following meanings... +With this sample command, it will .. - Scan all servers defined in config file -- Send scan resulsts to slack, email -- Only Reporting CVEs that CVSS score over 7 +- Send scan results to slack and email +- Only Reporting CVEs that CVSS score is over 7 - Print scan result to terminal -### Scan specified servers +### Scan specific servers ``` $ vuls scan server1 server2 ``` -Examples of the above are the following meanings... +With this sample command, it will .. - Scan only 2 servers. (server1, server2) - Print scan result to terminal @@ -523,14 +518,14 @@ Examples of the above are the following meanings... # Usage: Scan vulnerability of non-OS package -Vuls is possible to detect vulnerabilities something you compiled by yourself or library of language, framework that has been registered in the [CPE](https://nvd.nist.gov/cpe.cfm). +It is possible to detect vulnerabilities something you compiled by yourself or the language libraries and the frameworks that have been registered in the [CPE](https://nvd.nist.gov/cpe.cfm). - How to search CPE name by software name - [NVD: Search Common Platform Enumerations (CPE)](https://web.nvd.nist.gov/view/cpe/search) **Check CPE Naming Format: 2.2** - Configuration -If you want to detect the vulnerbility of Ruby on Rails v4.2.1, define as below. +If you want to detect the vulnerbility of Ruby on Rails v4.2.1, here is an example. ``` [servers] @@ -570,7 +565,7 @@ fetchnvd: $ go-cve-dictionary fetchnvd -entire ``` -- Fetch data last 2 years +- Fetch data of the last 2 years ``` $ go-cve-dictionary fetchnvd -last2y From 4b95d55ee67060af6219d30a08172efc5d509e08 Mon Sep 17 00:00:00 2001 From: hagiwara Date: Fri, 1 Apr 2016 15:11:03 +0900 Subject: [PATCH 2/3] =?UTF-8?q?=E5=BE=AE=E4=BF=AE=E6=AD=A3?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .README.md.un~ | Bin 0 -> 14362 bytes README.md | 24 +- README.md~ | 642 +++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 656 insertions(+), 10 deletions(-) create mode 100644 .README.md.un~ create mode 100644 README.md~ diff --git a/.README.md.un~ b/.README.md.un~ new file mode 100644 index 0000000000000000000000000000000000000000..7abab654aa5e8967dfe66184a8157f9873cd8680 GIT binary patch literal 14362 zcmeI3&yN&E6vt<00p&L+DnA6u?1hcl9%Wr&iRd7U2=c=StcfczJ-xj%?e=t!(>)7A z46@)w6BA?N#S7xW9~?FQOw@zk6r+jJgNb<1_!kgOjH};oPuK8R2^zC?tCzfe)jzA| zecr3?s_GpL>SvEcBk?bVVmxv2($LHouYdFBy2WF^P2GOx=ND_!ecylk!|n@D|MA@m z?-vTi(}hA|VPBsbcGTe`d$4g4+6$cz-3X=7YAuY;taxW$5BBsHp7@$aAq$|~Md5`D zvl9ON1?e8qJFT5V4~)(%-J9X)DO_5O`(6)S46r9*GbBUP875-64yuL7?u8b)!(yTr zObRf)n_((+Fl|R=_rd$mz!KBiK9!_?W1!L=pjH0yM(9Rv8pKf!kiZC63tmK$Hm$s5@doX-}B40QH#!#T?loGOdtfB9=v>5kzJ&35S?1 z0@LSL!*mMO%I6lb;Ca>+A}i}Hn@!pirYtc1E-^7fm`&5L5)-j35se_SGL)|!_Vt2J zFe;-n%buH!X+&U}zb6{^kLNuf*(?>!A-6I#<7ZP!c*aO^bY}Tx$%+rCGFUweThujX z@A5j?c!bHeWW-OrX3*eU;kFzmp)J!_!4j-}j0uOBEdu87Y?$mH&!Ae_KN8Dw(G()X zc?FIGZ2w3n7!_bTDly5-V$K~nTniGFw$iv6xS={0grN%kRJG%lN<24GY0XzttuXSN zZY2nVH1HD@cXo%d=Z3X7NpV?P<5>xROP0Z96zylBQx*~PrHq&|Dq@OH!}hS!Jh6%8MsDXqWEPY5hH(o_7p{hhi7hb^%PP?tBD0u; zL(CSjVswwhr2AjpD?U8O{jcKB`A7dN1A+l?6VJ+p!BK`D?O5fY^DpfY5n6 z=K+y-*Tj6Y_+bnI_5+)^&tj(+ObW2?%CHx&%ubDcQ`Qe~v{mudcpUjE=!~!?Vh)&s zsG6;jC3+l0e7A@O6YYLzFZWrT^nytNPG4s6K2&x9FG(8b8hlo8r=#|{t;nmX9d-8v z2F&|{sHNv2$%{+YTF8?8vQt93(0c~YI9IH#as~# zTaw{psOoW%gZ*H**-^WFPF$tFa#fPl1}6PBC$F-JD^+*BS`g`}3i4Yk+Do<`)bbQg zF;@i7laglzRgDXtYr%8SQG22!#U!F?`cJijreF7?G|As_2yC5bAYo2JEkNNEb437r zUIILYs_;PUJ`^_q;9ZUy@f%^>R@HdGoAL+71CL8OZnK?3B(z&aBMJ0XsD&t;Vy*~? zFH4ALQPnvCaU~!Qbs=)XKj%hH>qWcB@GYq2CY)lf2yU-PZf~Nh^MczFaJ$=4JL6~~ zsOHFsXoF}H@qGZb$b?hO6@lzEi45N)7d{loRs-1YFCZ4}KT z%1@vcpKyw~BJjN-@qLV{J{9;@0pC^!!w9#TszVssM*x_klUc_W65~avMMrNK7odAP zL)Y7xjrkEZ{v^;{{;z6CwG+)byC#%>#% z$|g)g*PB$S@75hPwl~Iq+T&edwVF;=!S`)$b-= z5O{8qcFs(2YPKu#>#h?wt0lKI*d9L7oERRu!>tCB@s`Q3y49^rwY^EF5mo#D$94Up zAJ8sjP_aL-{ebp}arvzOCNjPh8y{be>n!K_`s}H valE)*nrNl1ra#~|F?*N#URvTXv{VV>N~w-`^P8p4jhrW4r#_CD`1SnX1UAzP literal 0 HcmV?d00001 diff --git a/README.md b/README.md index a132ee23..c2c92763 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,7 @@ Vulnerability scanner for Linux, agentless, written in golang. [README in Japanese](https://github.com/future-architect/vuls/blob/master/README.ja.md) +We have a slack team. [Join slack team](http://goo.gl/forms/xm5KFo35tu) [![asciicast](https://asciinema.org/a/3y9zrf950agiko7klg8abvyck.png)](https://asciinema.org/a/3y9zrf950agiko7klg8abvyck) @@ -58,7 +59,10 @@ Vuls is a tool created to solve the problems listed above. It has the following ---- -# step-by-step explanation +# Hello Vuls + +This tutorial will let you scan the vulnerabilities on the localhost with vuls. +This can be done in the following steps. 1. Launch Amazon Linux 1. Enable to ssh from localhost @@ -70,7 +74,7 @@ Vuls is a tool created to solve the problems listed above. It has the following 1. Scan 1. TUI(Terminal-Based User Interface) -## 1. Launch Amazon Linux +## Step1. Launch Amazon Linux - We are using the old AMI (amzn-ami-hvm-2015.09.1.x86_64-gp2 - ami-383c1956) for this example - Instance size: t2.medium @@ -84,7 +88,7 @@ Vuls is a tool created to solve the problems listed above. It has the following repo_upgrade: none ``` -## 2. Generate a ssh-key on local machine +## Step2. Generate a ssh-key on local machine Create a keypair then append public key to authorized_keys ```bash @@ -92,7 +96,7 @@ $ ssh-keygen -t rsa $ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys ``` -## 3. Install requirements +## Step3. Install requirements Vuls requires the following packages. @@ -122,7 +126,7 @@ Set the OS environment variable to current shell $ source /etc/profile.d/goenv.sh ``` -## 4. Deploy go-cve-dictionary +## Step4. Deploy go-cve-dictionary go get @@ -153,7 +157,7 @@ $ go-cve-dictionary server [Mar 24 15:21:56] INFO Listening on 127.0.0.1:1323 ``` -## 5. Deploy vuls +## Step5. Deploy vuls Launch a new terminal, SSH to the ec2 instance. @@ -162,7 +166,7 @@ go get $ go get github.com/future-architect/vuls ``` -## 6. Config +## Step6. Config Create a config file(TOML format). @@ -177,13 +181,13 @@ user = "ec2-user" keyPath = "/home/ec2-user/.ssh/id_rsa" ``` -## 7. Initializing Vuls +## Step7. Initializing Vuls ``` $ vuls prepare ``` -## 8. Start Scanning +## Step8. Start Scanning ``` $ vuls scan @@ -216,7 +220,7 @@ Package/CPE java-1.7.0-openjdk-1.7.0.91-2.6.2.2.63.amzn1 -> java-1.7.0-openj ``` -## 9. TUI +## Step9. TUI Vuls has Terminal-Based User Interface to display the scan result. diff --git a/README.md~ b/README.md~ new file mode 100644 index 00000000..461dc564 --- /dev/null +++ b/README.md~ @@ -0,0 +1,642 @@ + +# Vuls: VULnerability Scanner + +[![Slack](https://img.shields.io/badge/slack-join-blue.svg)](http://goo.gl/forms/xm5KFo35tu) + +Vulnerability scanner for Linux, agentless, written in golang. + +[README in Japanese](https://github.com/future-architect/vuls/blob/master/README.ja.md) +We have a slack team. [Join slack team](http://goo.gl/forms/xm5KFo35tu) + +[![asciicast](https://asciinema.org/a/3y9zrf950agiko7klg8abvyck.png)](https://asciinema.org/a/3y9zrf950agiko7klg8abvyck) + +![Vuls-slack](img/vuls-slack-en.png) + + + +---- + +# Abstract + +For a system administrator, having to perform security vulnerability analysis and software update on a daily basis can be a burden. +To avoid downtime in production environment, it is common for system administrator to choose not to use the automatic update option provided by package manager and to perform update manually. +This leads to the following problems. +- System administrator will have to constantly watch out for any new vulnerabilities in NVD(National Vulnerability Database) and etc. +- It might be impossible for the system administrator to monitor all the software if there are a large number of software installed in server. +- It is expensive to perform anaylsis to determine the servers affected by new vulnerabilities. The possibility of overlooking a server or two during analysis is there. + + +Vuls is a tool created to solve the problems listed above. It has the following characteristics. +- Informs users of the vulnerabilities that are related to the system. +- Informs users of the servers that are affected. +- Vulnerability detection is done automatically to prevent any oversight. +- Report is generated on regular basis using CRON etc. to manage vulnerability. + +![Vuls-Motivation](img/vuls-motivation.png) + +---- + +# Main Features + +- Scan for any vulnerabilities in Linux Server + - Supports Ubuntu, Debian, CentOS, Amazon Linux, RHEL + - Cloud, on-premise, Docker +- Scan middleware that are not included in OS package management + - Scan middleware, programming language libraries and framework for vulnerability + - Support software registered in CPE +- Agentless architecture + - User is required to only setup one machine that is connected to other target servers via SSH +- Auto generation of configuration file template + - Auto detection of servers set using CIDR, generate configuration file template +- Email and Slack notification is possible (supports Japanese language) +- Scan result is viewable on accessory software, TUI Viewer terminal. + +---- + +# What Vuls Doesn't Do + +- Vuls doesn't update the vulnerable packages. + +---- + +# Hello Vuls + +This tutorial will let you scan the vulnerabilities on the localhost with vuls. +This can be done in the following steps. + +1. Launch Amazon Linux +1. Enable to ssh from localhost +1. Install requirements +1. Deploy go-cve-dictionary +1. Deploy Vuls +1. Configuration +1. Prepare +1. Scan +1. TUI(Terminal-Based User Interface) + +## 1. Launch Amazon Linux + +- We are using the old AMI (amzn-ami-hvm-2015.09.1.x86_64-gp2 - ami-383c1956) for this example +- Instance size: t2.medium + - For the first time, t2.medium and above is required for the data fetch from NVD + - You can switch to t2.nano after the initial data fetch. +- Add the following to the cloud-init, to avoid auto-update at the first launch. + + - [Q: How do I disable the automatic installation of critical and important security updates on initial launch?](https://aws.amazon.com/amazon-linux-ami/faqs/?nc1=h_ls) + ``` + #cloud-config + repo_upgrade: none + ``` + +## 2. Generate a ssh-key on local machine + +Create a keypair then append public key to authorized_keys +```bash +$ ssh-keygen -t rsa +$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys +``` + +## 3. Install requirements + +Vuls requires the following packages. + +- sqlite +- git +- gcc +- go v1.6 + - https://golang.org/doc/install + +```bash +$ ssh ec2-user@52.100.100.100 -i ~/.ssh/private.pem +$ sudo yum -y install sqlite git gcc +$ wget https://storage.googleapis.com/golang/go1.6.linux-amd64.tar.gz +$ sudo tar -C /usr/local -xzf go1.6.linux-amd64.tar.gz +$ mkdir $HOME/go +``` +Add these lines into /etc/profile.d/goenv.sh + +```bash +export GOROOT=/usr/local/go +export GOPATH=$HOME/go +export PATH=$PATH:$GOROOT/bin:$GOPATH/bin +``` + +Set the OS environment variable to current shell +```bash +$ source /etc/profile.d/goenv.sh +``` + +## 4. Deploy go-cve-dictionary + +go get + +```bash +$ sudo mkdir /var/log/vuls +$ sudo chown ec2-user /var/log/vuls +$ sudo chmod 700 /var/log/vuls +$ go get github.com/kotakanbe/go-cve-dictionary +``` + +Start go-cve-dictionary as server mode. +For the first time, go-cve-dictionary fetches vulnerability data from NVD. +It takes about 10 minutes (on AWS). + +```bash +$ go-cve-dictionary server +... Fetching ... +$ ls -alh cve.sqlite3 +-rw-r--r-- 1 ec2-user ec2-user 7.0M Mar 24 13:20 cve.sqlite3 +``` + +Now we successfully collected vulnerbility data, then start as server mode again. +```bash +$ go-cve-dictionary server +[Mar 24 15:21:55] INFO Opening DB. datafile: /home/ec2-user/cve.sqlite3 +[Mar 24 15:21:55] INFO Migrating DB +[Mar 24 15:21:56] INFO Starting HTTP Sever... +[Mar 24 15:21:56] INFO Listening on 127.0.0.1:1323 +``` + +## 5. Deploy vuls + +Launch a new terminal, SSH to the ec2 instance. + +go get +``` +$ go get github.com/future-architect/vuls +``` + +## 6. Config + +Create a config file(TOML format). + +``` +$ cat config.toml +[servers] + +[servers.172-31-4-82] +host = "172.31.4.82" +port = "22" +user = "ec2-user" +keyPath = "/home/ec2-user/.ssh/id_rsa" +``` + +## 7. Initializing Vuls + +``` +$ vuls prepare +``` + +## 8. Start Scanning + +``` +$ vuls scan +INFO[0000] Begin scannig (config: /home/ec2-user/config.toml) + +... snip ... + +172-31-4-82 (amazon 2015.09) +============================ +CVE-2016-0494 10.0 Unspecified vulnerability in the Java SE and Java SE Embedded components in Oracle + Java SE 6u105, 7u91, and 8u66 and Java SE Embedded 8u65 allows remote attackers to + affect confidentiality, integrity, and availability via unknown vectors related to + 2D. +... snip ... + +CVE-2016-0494 +------------- +Score 10.0 (High) +Vector (AV:N/AC:L/Au:N/C:C/I:C/A:C) +Summary Unspecified vulnerability in the Java SE and Java SE Embedded components in Oracle Java SE 6u105, + 7u91, and 8u66 and Java SE Embedded 8u65 allows remote attackers to affect confidentiality, + integrity, and availability via unknown vectors related to 2D. +NVD https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2016-0494 +MITRE https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-0494 +CVE Details http://www.cvedetails.com/cve/CVE-2016-0494 +CVSS Claculator https://nvd.nist.gov/cvss/v2-calculator?name=CVE-2016-0494&vector=(AV:N/AC:L/Au:N/C:C/I:C/A:C) +RHEL-CVE https://access.redhat.com/security/cve/CVE-2016-0494 +ALAS-2016-643 https://alas.aws.amazon.com/ALAS-2016-643.html +Package/CPE java-1.7.0-openjdk-1.7.0.91-2.6.2.2.63.amzn1 -> java-1.7.0-openjdk-1:1.7.0.95-2.6.4.0.65.amzn1 + +``` + +## 9. TUI + +Vuls has Terminal-Based User Interface to display the scan result. + +``` +$ vuls tui +``` + +![Vuls-TUI](img/hello-vuls-tui.png) + + +---- + +# Architecture + +![Vuls-Architecture](img/vuls-architecture.png) + +## go-cve-dictinary +- Fetch vulnerbility information from NVD, JVN(Japanese), then insert into SQLite. + +## Vuls +- Scan vulnerabilities of the servers and generate a list of the CVE ID +- In order to get more information of the detected CVE, send HTTP request to go-cve-dictinary +- Send a report by Slack, Email +- System operator can view the latest report by terminal + +---- + +# Use cases + +## Scan all servers + +![Vuls-Usecase1](img/vuls-usecase-elb-rails-rds-all.png) + +## Scan a single server + +web/app server in the same configuration under the load balancer + +![Vuls-Usecase2](img/vuls-usecase-elb-rails-rds-single.png) + +---- + +# Support OS + +| Distribution| Release | +|:------------|-------------------:| +| Ubuntu | 12, 14, 16| +| Debian | 7, 8| +| RHEL | 4, 5, 6, 7| +| CentOS | 5, 6, 7| +| Amazon Linux| All | + +---- + + +# Usage: Automatic Server Discovery + +Discovery subcommand discovers active servers specifed in CIDR range, then print the template of config file(TOML format) to terminal. + +``` +$ vuls discover -help +discover: + discover 192.168.0.0/24 +``` + +## Exapmle + +``` +$ vuls discover 172.31.4.0/24 +# Create config.toml using below and then ./vuls --config=/path/to/config.toml + +[slack] +hookURL = "https://hooks.slack.com/services/abc123/defghijklmnopqrstuvwxyz" +channel = "#channel-name" +#channel = "#{servername}" +iconEmoji = ":ghost:" +authUser = "username" +notifyUsers = ["@username"] + +[mail] +smtpAddr = "smtp.gmail.com" +smtpPort = 465 +user = "username" +password = "password" +from = "from@address.com" +to = ["to@address.com"] +cc = ["cc@address.com"] +subjectPrefix = "[vuls]" + +[default] +#port = "22" +#user = "username" +#password = "password" +#keyPath = "/home/username/.ssh/id_rsa" +#keyPassword = "password" + +[servers] + +[servers.172-31-4-82] +host = "172.31.4.82" +#port = "22" +#user = "root" +#password = "password" +#keyPath = "/home/username/.ssh/id_rsa" +#keyPassword = "password" +#cpeNames = [ +# "cpe:/a:rubyonrails:ruby_on_rails:4.2.1", +#] +``` + +You can customize your configuration using this template. + +---- + +# Configuration + +- Slack section + ``` + [slack] + hookURL = "https://hooks.slack.com/services/abc123/defghijklmnopqrstuvwxyz" + channel = "#channel-name" + #channel = "#{servername}" + iconEmoji = ":ghost:" + authUser = "username" + notifyUsers = ["@username"] + ``` + + - hookURL : Incomming webhook's URL + - channel : channel name. + If you set #{servername} to channel, the report will be sent to #servername channel. + In the following example, the report will be sent to the #server1 and #server2. + Be sure to create these channels before scanning. + ``` + [slack] + channel = "#{servername}" + ...snip... + + [servers] + + [servers.server1] + host = "172.31.4.82" + ...snip... + + [servers.server2] + host = "172.31.4.83" + ...snip... + ``` + + - iconEmoji: emoji + - authUser: username of the slack team + - notifyUsers: a list of Slack usernames to send Slack notifications. + If you set ["@foo", "@bar"] to notifyUsers, @foo @bar will be included in text. + So @foo, @bar can receive mobile push notifications on their smartphone. + +- Mail section + ``` + [mail] + smtpAddr = "smtp.gmail.com" + smtpPort = 465 + user = "username" + password = "password" + from = "from@address.com" + to = ["to@address.com"] + cc = ["cc@address.com"] + subjectPrefix = "[vuls]" + ``` + +- Defualt section + ``` + [default] + #port = "22" + #user = "username" + #password = "password" + #keyPath = "/home/username/.ssh/id_rsa" + #keyPassword = "password" + ``` + Items of the defualt section will be used if not specified. + +- servers section + ``` + [servers] + + [servers.172-31-4-82] + host = "172.31.4.82" + #port = "22" + #user = "root" + #password = "password" + #keyPath = "/home/username/.ssh/id_rsa" + #keyPassword = "password" + #cpeNames = [ + # "cpe:/a:rubyonrails:ruby_on_rails:4.2.1", + #] + ``` + You can overwrite the default value specified in default section. + Vuls supports multiple SSH authentication methods. + - SSH agent + - SSH public key authentication (with password, empty password) + - Password authentication + +---- + +# Usage: Prepare + +Prepare subcommand installs required packages on each server. + +| Distribution| Release | Requirements | +|:------------|-------------------:|:-------------| +| Ubuntu | 12, 14, 16| - | +| Debian | 7, 8| apptitude | +| CentOS | 5| yum-plugin-security, yum-changelog | +| CentOS | 6, 7| yum-plugin-security, yum-plugin-changelog | +| Amazon | All | - | +| RHEL | 4, 5, 6, 7 | - | + + +``` +$ vuls prepare -help +prepare: + prepare [-config=/path/to/config.toml] [-debug] + + -config string + /path/to/toml (default "$PWD/config.toml") + -debug + debug mode + -use-unattended-upgrades + [Depricated] For Ubuntu, install unattended-upgrades +``` + +---- + +# Usage: Scan + +``` +$ vuls scan -help +scan: + scan + [-lang=en|ja] + [-config=/path/to/config.toml] + [-dbpath=/path/to/vuls.sqlite3] + [-cve-dictionary-url=http://127.0.0.1:1323] + [-cvss-over=7] + [-report-slack] + [-report-mail] + [-http-proxy=http://192.168.0.1:8080] + [-debug] + [-debug-sql] + -config string + /path/to/toml (default "$PWD/config.toml") + -cve-dictionary-url string + http://CVE.Dictionary (default "http://127.0.0.1:1323") + -cvss-over float + -cvss-over=6.5 means reporting CVSS Score 6.5 and over (default: 0 (means report all)) + -dbpath string + /path/to/sqlite3 (default "$PWD/vuls.sqlite3") + -debug + debug mode + -debug-sql + SQL debug mode + -http-proxy string + http://proxy-url:port (default: empty) + -lang string + [en|ja] (default "en") + -report-mail + Email report + -report-slack + Slack report + -use-unattended-upgrades + [Depricated] For Ubuntu. Scan by unattended-upgrades or not (use apt-get upgrade --dry-run by default) + -use-yum-plugin-security + [Depricated] For CentOS 5. Scan by yum-plugin-security or not (use yum check-update by default) + +``` + +## example + +Run go-cve-dictionary as server mode before scanning. +``` +$ go-cve-dictionary server +``` + +### Scan all servers defined in config file +``` +$ vuls scan --report-slack --report-mail --cvss-over=7 +``` +With this sample command, it will .. +- Scan all servers defined in config file +- Send scan results to slack and email +- Only Reporting CVEs that CVSS score is over 7 +- Print scan result to terminal + +### Scan specific servers +``` +$ vuls scan server1 server2 +``` +With this sample command, it will .. +- Scan only 2 servers. (server1, server2) +- Print scan result to terminal + +---- + +# Usage: Scan vulnerability of non-OS package + +It is possible to detect vulnerabilities something you compiled by yourself or the language libraries and the frameworks that have been registered in the [CPE](https://nvd.nist.gov/cpe.cfm). + +- How to search CPE name by software name + - [NVD: Search Common Platform Enumerations (CPE)](https://web.nvd.nist.gov/view/cpe/search) + **Check CPE Naming Format: 2.2** + +- Configuration +If you want to detect the vulnerbility of Ruby on Rails v4.2.1, here is an example. + ``` + [servers] + + [servers.172-31-4-82] + host = "172.31.4.82" + user = "ec2-user" + keyPath = "/home/username/.ssh/id_rsa" + cpeNames = [ + "cpe:/a:rubyonrails:ruby_on_rails:4.2.1", + ] + ``` + +# Usage: Update NVD Data. + +``` +$ go-cve-dictionary fetchnvd -h +fetchnvd: + fetchnvd + [-last2y] + [-dbpath=/path/to/cve.sqlite3] + [-debug] + [-debug-sql] + + -dbpath string + /path/to/sqlite3 (default "$PWD/cve.sqlite3") + -debug + debug mode + -debug-sql + SQL debug mode + -last2y + Refresh NVD data in the last two years. +``` + +- Fetch data of the entire period + +``` +$ go-cve-dictionary fetchnvd -entire +``` + +- Fetch data of the last 2 years + +``` +$ go-cve-dictionary fetchnvd -last2y +``` + +---- + +# Misc + +- HTTP Proxy Support +If your system is behind HTTP proxy, you have to specify --http-proxy option. + +- How to Daemonize go-cve-dictionary +Use Systemd, Upstart or supervisord, daemontools... + +- How to update vulnerbility data automatically. +Use job scheduler like Cron (with -last2y option). + +- How to cross compile + ```bash + $ cd /path/to/your/local-git-reporsitory/vuls + $ GOOS=linux GOARCH=amd64 go build -o vuls.amd64 + ``` + +- Logging +Log wrote to under /var/log/vuls/ + +- Debug +Run with --debug, --sql-debug option. + +- Windows +Use Microsoft Baseline Secuirty Analyzer. [MBSA](https://technet.microsoft.com/en-us/security/cc184924.aspx) + +---- + +# Data Source + +- [NVD](https://nvd.nist.gov/) +- [JVN(Japanese)](http://jvndb.jvn.jp/apis/myjvn/) + + +# Authors + +kotakanbe ([@kotakanbe](https://twitter.com/kotakanbe)) created vuls and [these fine people](https://github.com/future-architect/vuls/graphs/contributors) have contributed. + +---- + +# Contribute + +1. Fork it +2. Create your feature branch (`git checkout -b my-new-feature`) +3. Commit your changes (`git commit -am 'Add some feature'`) +4. Push to the branch (`git push origin my-new-feature`) +5. Create new Pull Request + +---- + +# Change Log + +Please see [CHANGELOG](https://github.com/future-architect/vuls/blob/master/CHANGELOG.md). + +---- + +# Licence + +Please see [LICENSE](https://github.com/future-architect/vuls/blob/master/LICENSE). + + +[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/future-architect/vuls/trend.png)](https://bitdeli.com/free "Bitdeli Badge") + From 317ea7485d100d01afbdf7e27f66b6959887f679 Mon Sep 17 00:00:00 2001 From: hagiwara Date: Fri, 1 Apr 2016 16:13:26 +0900 Subject: [PATCH 3/3] =?UTF-8?q?=E5=BE=AE=E4=BF=AE=E6=AD=A3?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .README.md.un~ | Bin 14362 -> 0 bytes README.md | 20 +- README.md~ | 642 ------------------------------------------------- 3 files changed, 11 insertions(+), 651 deletions(-) delete mode 100644 .README.md.un~ delete mode 100644 README.md~ diff --git a/.README.md.un~ b/.README.md.un~ deleted file mode 100644 index 7abab654aa5e8967dfe66184a8157f9873cd8680..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 14362 zcmeI3&yN&E6vt<00p&L+DnA6u?1hcl9%Wr&iRd7U2=c=StcfczJ-xj%?e=t!(>)7A z46@)w6BA?N#S7xW9~?FQOw@zk6r+jJgNb<1_!kgOjH};oPuK8R2^zC?tCzfe)jzA| zecr3?s_GpL>SvEcBk?bVVmxv2($LHouYdFBy2WF^P2GOx=ND_!ecylk!|n@D|MA@m z?-vTi(}hA|VPBsbcGTe`d$4g4+6$cz-3X=7YAuY;taxW$5BBsHp7@$aAq$|~Md5`D zvl9ON1?e8qJFT5V4~)(%-J9X)DO_5O`(6)S46r9*GbBUP875-64yuL7?u8b)!(yTr zObRf)n_((+Fl|R=_rd$mz!KBiK9!_?W1!L=pjH0yM(9Rv8pKf!kiZC63tmK$Hm$s5@doX-}B40QH#!#T?loGOdtfB9=v>5kzJ&35S?1 z0@LSL!*mMO%I6lb;Ca>+A}i}Hn@!pirYtc1E-^7fm`&5L5)-j35se_SGL)|!_Vt2J zFe;-n%buH!X+&U}zb6{^kLNuf*(?>!A-6I#<7ZP!c*aO^bY}Tx$%+rCGFUweThujX z@A5j?c!bHeWW-OrX3*eU;kFzmp)J!_!4j-}j0uOBEdu87Y?$mH&!Ae_KN8Dw(G()X zc?FIGZ2w3n7!_bTDly5-V$K~nTniGFw$iv6xS={0grN%kRJG%lN<24GY0XzttuXSN zZY2nVH1HD@cXo%d=Z3X7NpV?P<5>xROP0Z96zylBQx*~PrHq&|Dq@OH!}hS!Jh6%8MsDXqWEPY5hH(o_7p{hhi7hb^%PP?tBD0u; zL(CSjVswwhr2AjpD?U8O{jcKB`A7dN1A+l?6VJ+p!BK`D?O5fY^DpfY5n6 z=K+y-*Tj6Y_+bnI_5+)^&tj(+ObW2?%CHx&%ubDcQ`Qe~v{mudcpUjE=!~!?Vh)&s zsG6;jC3+l0e7A@O6YYLzFZWrT^nytNPG4s6K2&x9FG(8b8hlo8r=#|{t;nmX9d-8v z2F&|{sHNv2$%{+YTF8?8vQt93(0c~YI9IH#as~# zTaw{psOoW%gZ*H**-^WFPF$tFa#fPl1}6PBC$F-JD^+*BS`g`}3i4Yk+Do<`)bbQg zF;@i7laglzRgDXtYr%8SQG22!#U!F?`cJijreF7?G|As_2yC5bAYo2JEkNNEb437r zUIILYs_;PUJ`^_q;9ZUy@f%^>R@HdGoAL+71CL8OZnK?3B(z&aBMJ0XsD&t;Vy*~? zFH4ALQPnvCaU~!Qbs=)XKj%hH>qWcB@GYq2CY)lf2yU-PZf~Nh^MczFaJ$=4JL6~~ zsOHFsXoF}H@qGZb$b?hO6@lzEi45N)7d{loRs-1YFCZ4}KT z%1@vcpKyw~BJjN-@qLV{J{9;@0pC^!!w9#TszVssM*x_klUc_W65~avMMrNK7odAP zL)Y7xjrkEZ{v^;{{;z6CwG+)byC#%>#% z$|g)g*PB$S@75hPwl~Iq+T&edwVF;=!S`)$b-= z5O{8qcFs(2YPKu#>#h?wt0lKI*d9L7oERRu!>tCB@s`Q3y49^rwY^EF5mo#D$94Up zAJ8sjP_aL-{ebp}arvzOCNjPh8y{be>n!K_`s}H valE)*nrNl1ra#~|F?*N#URvTXv{VV>N~w-`^P8p4jhrW4r#_CD`1SnX1UAzP diff --git a/README.md b/README.md index c2c92763..7ac9c5b1 100644 --- a/README.md +++ b/README.md @@ -88,7 +88,9 @@ This can be done in the following steps. repo_upgrade: none ``` -## Step2. Generate a ssh-key on local machine +## Step2. SSH setting + +This is required to ssh to itself. Create a keypair then append public key to authorized_keys ```bash @@ -181,7 +183,7 @@ user = "ec2-user" keyPath = "/home/ec2-user/.ssh/id_rsa" ``` -## Step7. Initializing Vuls +## Step7. Setting up target servers for vuls ``` $ vuls prepare @@ -241,14 +243,14 @@ $ vuls tui - Fetch vulnerbility information from NVD, JVN(Japanese), then insert into SQLite. ## Vuls -- Scan vulnerabilities of the servers and generate a list of the CVE ID -- In order to get more information of the detected CVE, send HTTP request to go-cve-dictinary +- Scan vulnerabilities on the servers and create a list of the CVE ID +- For more detailed information of the detected CVE, send HTTP request to go-cve-dictinary - Send a report by Slack, Email - System operator can view the latest report by terminal ---- -# Use cases +# Use Cases ## Scan all servers @@ -507,7 +509,7 @@ $ vuls scan --report-slack --report-mail --cvss-over=7 With this sample command, it will .. - Scan all servers defined in config file - Send scan results to slack and email -- Only Reporting CVEs that CVSS score is over 7 +- Only Report CVEs that CVSS score is over 7 - Print scan result to terminal ### Scan specific servers @@ -522,14 +524,14 @@ With this sample command, it will .. # Usage: Scan vulnerability of non-OS package -It is possible to detect vulnerabilities something you compiled by yourself or the language libraries and the frameworks that have been registered in the [CPE](https://nvd.nist.gov/cpe.cfm). +It is possible to detect vulnerabilities something you compiled by yourself, the language libraries and the frameworks that have been registered in the [CPE](https://nvd.nist.gov/cpe.cfm). - How to search CPE name by software name - [NVD: Search Common Platform Enumerations (CPE)](https://web.nvd.nist.gov/view/cpe/search) **Check CPE Naming Format: 2.2** - Configuration -If you want to detect the vulnerbility of Ruby on Rails v4.2.1, here is an example. +To detect the vulnerbility of Ruby on Rails v4.2.1, cpeNames needs to be set in the servers section. ``` [servers] @@ -569,7 +571,7 @@ fetchnvd: $ go-cve-dictionary fetchnvd -entire ``` -- Fetch data of the last 2 years +- Fetch data in the last 2 years ``` $ go-cve-dictionary fetchnvd -last2y diff --git a/README.md~ b/README.md~ deleted file mode 100644 index 461dc564..00000000 --- a/README.md~ +++ /dev/null @@ -1,642 +0,0 @@ - -# Vuls: VULnerability Scanner - -[![Slack](https://img.shields.io/badge/slack-join-blue.svg)](http://goo.gl/forms/xm5KFo35tu) - -Vulnerability scanner for Linux, agentless, written in golang. - -[README in Japanese](https://github.com/future-architect/vuls/blob/master/README.ja.md) -We have a slack team. [Join slack team](http://goo.gl/forms/xm5KFo35tu) - -[![asciicast](https://asciinema.org/a/3y9zrf950agiko7klg8abvyck.png)](https://asciinema.org/a/3y9zrf950agiko7klg8abvyck) - -![Vuls-slack](img/vuls-slack-en.png) - - - ----- - -# Abstract - -For a system administrator, having to perform security vulnerability analysis and software update on a daily basis can be a burden. -To avoid downtime in production environment, it is common for system administrator to choose not to use the automatic update option provided by package manager and to perform update manually. -This leads to the following problems. -- System administrator will have to constantly watch out for any new vulnerabilities in NVD(National Vulnerability Database) and etc. -- It might be impossible for the system administrator to monitor all the software if there are a large number of software installed in server. -- It is expensive to perform anaylsis to determine the servers affected by new vulnerabilities. The possibility of overlooking a server or two during analysis is there. - - -Vuls is a tool created to solve the problems listed above. It has the following characteristics. -- Informs users of the vulnerabilities that are related to the system. -- Informs users of the servers that are affected. -- Vulnerability detection is done automatically to prevent any oversight. -- Report is generated on regular basis using CRON etc. to manage vulnerability. - -![Vuls-Motivation](img/vuls-motivation.png) - ----- - -# Main Features - -- Scan for any vulnerabilities in Linux Server - - Supports Ubuntu, Debian, CentOS, Amazon Linux, RHEL - - Cloud, on-premise, Docker -- Scan middleware that are not included in OS package management - - Scan middleware, programming language libraries and framework for vulnerability - - Support software registered in CPE -- Agentless architecture - - User is required to only setup one machine that is connected to other target servers via SSH -- Auto generation of configuration file template - - Auto detection of servers set using CIDR, generate configuration file template -- Email and Slack notification is possible (supports Japanese language) -- Scan result is viewable on accessory software, TUI Viewer terminal. - ----- - -# What Vuls Doesn't Do - -- Vuls doesn't update the vulnerable packages. - ----- - -# Hello Vuls - -This tutorial will let you scan the vulnerabilities on the localhost with vuls. -This can be done in the following steps. - -1. Launch Amazon Linux -1. Enable to ssh from localhost -1. Install requirements -1. Deploy go-cve-dictionary -1. Deploy Vuls -1. Configuration -1. Prepare -1. Scan -1. TUI(Terminal-Based User Interface) - -## 1. Launch Amazon Linux - -- We are using the old AMI (amzn-ami-hvm-2015.09.1.x86_64-gp2 - ami-383c1956) for this example -- Instance size: t2.medium - - For the first time, t2.medium and above is required for the data fetch from NVD - - You can switch to t2.nano after the initial data fetch. -- Add the following to the cloud-init, to avoid auto-update at the first launch. - - - [Q: How do I disable the automatic installation of critical and important security updates on initial launch?](https://aws.amazon.com/amazon-linux-ami/faqs/?nc1=h_ls) - ``` - #cloud-config - repo_upgrade: none - ``` - -## 2. Generate a ssh-key on local machine - -Create a keypair then append public key to authorized_keys -```bash -$ ssh-keygen -t rsa -$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys -``` - -## 3. Install requirements - -Vuls requires the following packages. - -- sqlite -- git -- gcc -- go v1.6 - - https://golang.org/doc/install - -```bash -$ ssh ec2-user@52.100.100.100 -i ~/.ssh/private.pem -$ sudo yum -y install sqlite git gcc -$ wget https://storage.googleapis.com/golang/go1.6.linux-amd64.tar.gz -$ sudo tar -C /usr/local -xzf go1.6.linux-amd64.tar.gz -$ mkdir $HOME/go -``` -Add these lines into /etc/profile.d/goenv.sh - -```bash -export GOROOT=/usr/local/go -export GOPATH=$HOME/go -export PATH=$PATH:$GOROOT/bin:$GOPATH/bin -``` - -Set the OS environment variable to current shell -```bash -$ source /etc/profile.d/goenv.sh -``` - -## 4. Deploy go-cve-dictionary - -go get - -```bash -$ sudo mkdir /var/log/vuls -$ sudo chown ec2-user /var/log/vuls -$ sudo chmod 700 /var/log/vuls -$ go get github.com/kotakanbe/go-cve-dictionary -``` - -Start go-cve-dictionary as server mode. -For the first time, go-cve-dictionary fetches vulnerability data from NVD. -It takes about 10 minutes (on AWS). - -```bash -$ go-cve-dictionary server -... Fetching ... -$ ls -alh cve.sqlite3 --rw-r--r-- 1 ec2-user ec2-user 7.0M Mar 24 13:20 cve.sqlite3 -``` - -Now we successfully collected vulnerbility data, then start as server mode again. -```bash -$ go-cve-dictionary server -[Mar 24 15:21:55] INFO Opening DB. datafile: /home/ec2-user/cve.sqlite3 -[Mar 24 15:21:55] INFO Migrating DB -[Mar 24 15:21:56] INFO Starting HTTP Sever... -[Mar 24 15:21:56] INFO Listening on 127.0.0.1:1323 -``` - -## 5. Deploy vuls - -Launch a new terminal, SSH to the ec2 instance. - -go get -``` -$ go get github.com/future-architect/vuls -``` - -## 6. Config - -Create a config file(TOML format). - -``` -$ cat config.toml -[servers] - -[servers.172-31-4-82] -host = "172.31.4.82" -port = "22" -user = "ec2-user" -keyPath = "/home/ec2-user/.ssh/id_rsa" -``` - -## 7. Initializing Vuls - -``` -$ vuls prepare -``` - -## 8. Start Scanning - -``` -$ vuls scan -INFO[0000] Begin scannig (config: /home/ec2-user/config.toml) - -... snip ... - -172-31-4-82 (amazon 2015.09) -============================ -CVE-2016-0494 10.0 Unspecified vulnerability in the Java SE and Java SE Embedded components in Oracle - Java SE 6u105, 7u91, and 8u66 and Java SE Embedded 8u65 allows remote attackers to - affect confidentiality, integrity, and availability via unknown vectors related to - 2D. -... snip ... - -CVE-2016-0494 -------------- -Score 10.0 (High) -Vector (AV:N/AC:L/Au:N/C:C/I:C/A:C) -Summary Unspecified vulnerability in the Java SE and Java SE Embedded components in Oracle Java SE 6u105, - 7u91, and 8u66 and Java SE Embedded 8u65 allows remote attackers to affect confidentiality, - integrity, and availability via unknown vectors related to 2D. -NVD https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2016-0494 -MITRE https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-0494 -CVE Details http://www.cvedetails.com/cve/CVE-2016-0494 -CVSS Claculator https://nvd.nist.gov/cvss/v2-calculator?name=CVE-2016-0494&vector=(AV:N/AC:L/Au:N/C:C/I:C/A:C) -RHEL-CVE https://access.redhat.com/security/cve/CVE-2016-0494 -ALAS-2016-643 https://alas.aws.amazon.com/ALAS-2016-643.html -Package/CPE java-1.7.0-openjdk-1.7.0.91-2.6.2.2.63.amzn1 -> java-1.7.0-openjdk-1:1.7.0.95-2.6.4.0.65.amzn1 - -``` - -## 9. TUI - -Vuls has Terminal-Based User Interface to display the scan result. - -``` -$ vuls tui -``` - -![Vuls-TUI](img/hello-vuls-tui.png) - - ----- - -# Architecture - -![Vuls-Architecture](img/vuls-architecture.png) - -## go-cve-dictinary -- Fetch vulnerbility information from NVD, JVN(Japanese), then insert into SQLite. - -## Vuls -- Scan vulnerabilities of the servers and generate a list of the CVE ID -- In order to get more information of the detected CVE, send HTTP request to go-cve-dictinary -- Send a report by Slack, Email -- System operator can view the latest report by terminal - ----- - -# Use cases - -## Scan all servers - -![Vuls-Usecase1](img/vuls-usecase-elb-rails-rds-all.png) - -## Scan a single server - -web/app server in the same configuration under the load balancer - -![Vuls-Usecase2](img/vuls-usecase-elb-rails-rds-single.png) - ----- - -# Support OS - -| Distribution| Release | -|:------------|-------------------:| -| Ubuntu | 12, 14, 16| -| Debian | 7, 8| -| RHEL | 4, 5, 6, 7| -| CentOS | 5, 6, 7| -| Amazon Linux| All | - ----- - - -# Usage: Automatic Server Discovery - -Discovery subcommand discovers active servers specifed in CIDR range, then print the template of config file(TOML format) to terminal. - -``` -$ vuls discover -help -discover: - discover 192.168.0.0/24 -``` - -## Exapmle - -``` -$ vuls discover 172.31.4.0/24 -# Create config.toml using below and then ./vuls --config=/path/to/config.toml - -[slack] -hookURL = "https://hooks.slack.com/services/abc123/defghijklmnopqrstuvwxyz" -channel = "#channel-name" -#channel = "#{servername}" -iconEmoji = ":ghost:" -authUser = "username" -notifyUsers = ["@username"] - -[mail] -smtpAddr = "smtp.gmail.com" -smtpPort = 465 -user = "username" -password = "password" -from = "from@address.com" -to = ["to@address.com"] -cc = ["cc@address.com"] -subjectPrefix = "[vuls]" - -[default] -#port = "22" -#user = "username" -#password = "password" -#keyPath = "/home/username/.ssh/id_rsa" -#keyPassword = "password" - -[servers] - -[servers.172-31-4-82] -host = "172.31.4.82" -#port = "22" -#user = "root" -#password = "password" -#keyPath = "/home/username/.ssh/id_rsa" -#keyPassword = "password" -#cpeNames = [ -# "cpe:/a:rubyonrails:ruby_on_rails:4.2.1", -#] -``` - -You can customize your configuration using this template. - ----- - -# Configuration - -- Slack section - ``` - [slack] - hookURL = "https://hooks.slack.com/services/abc123/defghijklmnopqrstuvwxyz" - channel = "#channel-name" - #channel = "#{servername}" - iconEmoji = ":ghost:" - authUser = "username" - notifyUsers = ["@username"] - ``` - - - hookURL : Incomming webhook's URL - - channel : channel name. - If you set #{servername} to channel, the report will be sent to #servername channel. - In the following example, the report will be sent to the #server1 and #server2. - Be sure to create these channels before scanning. - ``` - [slack] - channel = "#{servername}" - ...snip... - - [servers] - - [servers.server1] - host = "172.31.4.82" - ...snip... - - [servers.server2] - host = "172.31.4.83" - ...snip... - ``` - - - iconEmoji: emoji - - authUser: username of the slack team - - notifyUsers: a list of Slack usernames to send Slack notifications. - If you set ["@foo", "@bar"] to notifyUsers, @foo @bar will be included in text. - So @foo, @bar can receive mobile push notifications on their smartphone. - -- Mail section - ``` - [mail] - smtpAddr = "smtp.gmail.com" - smtpPort = 465 - user = "username" - password = "password" - from = "from@address.com" - to = ["to@address.com"] - cc = ["cc@address.com"] - subjectPrefix = "[vuls]" - ``` - -- Defualt section - ``` - [default] - #port = "22" - #user = "username" - #password = "password" - #keyPath = "/home/username/.ssh/id_rsa" - #keyPassword = "password" - ``` - Items of the defualt section will be used if not specified. - -- servers section - ``` - [servers] - - [servers.172-31-4-82] - host = "172.31.4.82" - #port = "22" - #user = "root" - #password = "password" - #keyPath = "/home/username/.ssh/id_rsa" - #keyPassword = "password" - #cpeNames = [ - # "cpe:/a:rubyonrails:ruby_on_rails:4.2.1", - #] - ``` - You can overwrite the default value specified in default section. - Vuls supports multiple SSH authentication methods. - - SSH agent - - SSH public key authentication (with password, empty password) - - Password authentication - ----- - -# Usage: Prepare - -Prepare subcommand installs required packages on each server. - -| Distribution| Release | Requirements | -|:------------|-------------------:|:-------------| -| Ubuntu | 12, 14, 16| - | -| Debian | 7, 8| apptitude | -| CentOS | 5| yum-plugin-security, yum-changelog | -| CentOS | 6, 7| yum-plugin-security, yum-plugin-changelog | -| Amazon | All | - | -| RHEL | 4, 5, 6, 7 | - | - - -``` -$ vuls prepare -help -prepare: - prepare [-config=/path/to/config.toml] [-debug] - - -config string - /path/to/toml (default "$PWD/config.toml") - -debug - debug mode - -use-unattended-upgrades - [Depricated] For Ubuntu, install unattended-upgrades -``` - ----- - -# Usage: Scan - -``` -$ vuls scan -help -scan: - scan - [-lang=en|ja] - [-config=/path/to/config.toml] - [-dbpath=/path/to/vuls.sqlite3] - [-cve-dictionary-url=http://127.0.0.1:1323] - [-cvss-over=7] - [-report-slack] - [-report-mail] - [-http-proxy=http://192.168.0.1:8080] - [-debug] - [-debug-sql] - -config string - /path/to/toml (default "$PWD/config.toml") - -cve-dictionary-url string - http://CVE.Dictionary (default "http://127.0.0.1:1323") - -cvss-over float - -cvss-over=6.5 means reporting CVSS Score 6.5 and over (default: 0 (means report all)) - -dbpath string - /path/to/sqlite3 (default "$PWD/vuls.sqlite3") - -debug - debug mode - -debug-sql - SQL debug mode - -http-proxy string - http://proxy-url:port (default: empty) - -lang string - [en|ja] (default "en") - -report-mail - Email report - -report-slack - Slack report - -use-unattended-upgrades - [Depricated] For Ubuntu. Scan by unattended-upgrades or not (use apt-get upgrade --dry-run by default) - -use-yum-plugin-security - [Depricated] For CentOS 5. Scan by yum-plugin-security or not (use yum check-update by default) - -``` - -## example - -Run go-cve-dictionary as server mode before scanning. -``` -$ go-cve-dictionary server -``` - -### Scan all servers defined in config file -``` -$ vuls scan --report-slack --report-mail --cvss-over=7 -``` -With this sample command, it will .. -- Scan all servers defined in config file -- Send scan results to slack and email -- Only Reporting CVEs that CVSS score is over 7 -- Print scan result to terminal - -### Scan specific servers -``` -$ vuls scan server1 server2 -``` -With this sample command, it will .. -- Scan only 2 servers. (server1, server2) -- Print scan result to terminal - ----- - -# Usage: Scan vulnerability of non-OS package - -It is possible to detect vulnerabilities something you compiled by yourself or the language libraries and the frameworks that have been registered in the [CPE](https://nvd.nist.gov/cpe.cfm). - -- How to search CPE name by software name - - [NVD: Search Common Platform Enumerations (CPE)](https://web.nvd.nist.gov/view/cpe/search) - **Check CPE Naming Format: 2.2** - -- Configuration -If you want to detect the vulnerbility of Ruby on Rails v4.2.1, here is an example. - ``` - [servers] - - [servers.172-31-4-82] - host = "172.31.4.82" - user = "ec2-user" - keyPath = "/home/username/.ssh/id_rsa" - cpeNames = [ - "cpe:/a:rubyonrails:ruby_on_rails:4.2.1", - ] - ``` - -# Usage: Update NVD Data. - -``` -$ go-cve-dictionary fetchnvd -h -fetchnvd: - fetchnvd - [-last2y] - [-dbpath=/path/to/cve.sqlite3] - [-debug] - [-debug-sql] - - -dbpath string - /path/to/sqlite3 (default "$PWD/cve.sqlite3") - -debug - debug mode - -debug-sql - SQL debug mode - -last2y - Refresh NVD data in the last two years. -``` - -- Fetch data of the entire period - -``` -$ go-cve-dictionary fetchnvd -entire -``` - -- Fetch data of the last 2 years - -``` -$ go-cve-dictionary fetchnvd -last2y -``` - ----- - -# Misc - -- HTTP Proxy Support -If your system is behind HTTP proxy, you have to specify --http-proxy option. - -- How to Daemonize go-cve-dictionary -Use Systemd, Upstart or supervisord, daemontools... - -- How to update vulnerbility data automatically. -Use job scheduler like Cron (with -last2y option). - -- How to cross compile - ```bash - $ cd /path/to/your/local-git-reporsitory/vuls - $ GOOS=linux GOARCH=amd64 go build -o vuls.amd64 - ``` - -- Logging -Log wrote to under /var/log/vuls/ - -- Debug -Run with --debug, --sql-debug option. - -- Windows -Use Microsoft Baseline Secuirty Analyzer. [MBSA](https://technet.microsoft.com/en-us/security/cc184924.aspx) - ----- - -# Data Source - -- [NVD](https://nvd.nist.gov/) -- [JVN(Japanese)](http://jvndb.jvn.jp/apis/myjvn/) - - -# Authors - -kotakanbe ([@kotakanbe](https://twitter.com/kotakanbe)) created vuls and [these fine people](https://github.com/future-architect/vuls/graphs/contributors) have contributed. - ----- - -# Contribute - -1. Fork it -2. Create your feature branch (`git checkout -b my-new-feature`) -3. Commit your changes (`git commit -am 'Add some feature'`) -4. Push to the branch (`git push origin my-new-feature`) -5. Create new Pull Request - ----- - -# Change Log - -Please see [CHANGELOG](https://github.com/future-architect/vuls/blob/master/CHANGELOG.md). - ----- - -# Licence - -Please see [LICENSE](https://github.com/future-architect/vuls/blob/master/LICENSE). - - -[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/future-architect/vuls/trend.png)](https://bitdeli.com/free "Bitdeli Badge") -