Update CHANGELOG.md

Supports yum --enablerepo option (supports only base,updates for now)

.gitignore

@@ -1,9 +1,15 @@
+vuls
 .vscode
+*.txt
+*.json
+*.sqlite3*
+*.db
+tags
+.gitmodules
 coverage.out
 issues/
-*.txt
 vendor/
 log/
-.gitmodules
-vuls
-*.sqlite3
+results/
+*config.toml
+!setup/docker/*

CHANGELOG.md

@@ -1,5 +1,255 @@
 # Change Log
 
+## [v0.1.7](https://github.com/future-architect/vuls/tree/v0.1.7) (2016-11-08)
+[Full Changelog](https://github.com/future-architect/vuls/compare/v0.1.6...v0.1.7)
+
+**Implemented enhancements:**
+
+- Enable to scan only docker container, without docker host [\#122](https://github.com/future-architect/vuls/issues/122)
+- Add -skip-broken option \[CentOS only\] \#245 [\#248](https://github.com/future-architect/vuls/pull/248) ([kotakanbe](https://github.com/kotakanbe))
+- Display unknown CVEs to TUI [\#244](https://github.com/future-architect/vuls/pull/244) ([kotakanbe](https://github.com/kotakanbe))
+- Add the XML output [\#240](https://github.com/future-architect/vuls/pull/240) ([gleentea](https://github.com/gleentea))
+- add '-ssh-external' option to prepare subcommand [\#234](https://github.com/future-architect/vuls/pull/234) ([mykstmhr](https://github.com/mykstmhr))
+- Integrate OWASP Dependency Check [\#232](https://github.com/future-architect/vuls/pull/232) ([kotakanbe](https://github.com/kotakanbe))
+- Add support for reading CVE data from MySQL. [\#225](https://github.com/future-architect/vuls/pull/225) ([oswell](https://github.com/oswell))
+- Remove base docker image, -v shows commit hash [\#223](https://github.com/future-architect/vuls/pull/223) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+- Support ignore CveIDs in config [\#222](https://github.com/future-architect/vuls/pull/222) ([kotakanbe](https://github.com/kotakanbe))
+- Confirm before installing dependencies on prepare [\#219](https://github.com/future-architect/vuls/pull/219) ([kotakanbe](https://github.com/kotakanbe))
+- Remove all.json [\#218](https://github.com/future-architect/vuls/pull/218) ([kotakanbe](https://github.com/kotakanbe))
+- Add GitHub issue template [\#217](https://github.com/future-architect/vuls/pull/217) ([kotakanbe](https://github.com/kotakanbe))
+- Improve makefile, -version shows git hash, fix README [\#216](https://github.com/future-architect/vuls/pull/216) ([kotakanbe](https://github.com/kotakanbe))
+- change e-mail package from gomail to net/smtp [\#211](https://github.com/future-architect/vuls/pull/211) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+- Add only-containers option to scan subcommand \#122 [\#190](https://github.com/future-architect/vuls/pull/190) ([kotakanbe](https://github.com/kotakanbe))
+- Fix -results-dir option of scan subcommand [\#185](https://github.com/future-architect/vuls/pull/185) ([kotakanbe](https://github.com/kotakanbe))
+- Show error when no scannable servers are detected. [\#177](https://github.com/future-architect/vuls/pull/177) ([kotakanbe](https://github.com/kotakanbe))
+- Add sudo check to prepare subcommand [\#176](https://github.com/future-architect/vuls/pull/176) ([kotakanbe](https://github.com/kotakanbe))
+- Supports yum --enablerepo option \(supports only base,updates for now\) [\#147](https://github.com/future-architect/vuls/pull/147) ([kotakanbe](https://github.com/kotakanbe))
+
+**Fixed bugs:**
+
+- Debian 8.6 \(jessie\) scan does not show vulnerable packages [\#235](https://github.com/future-architect/vuls/issues/235)
+- panic: runtime error: index out of range - ubuntu 16.04 + vuls history [\#180](https://github.com/future-architect/vuls/issues/180)
+- Moved golang.org/x/net/context to context [\#243](https://github.com/future-architect/vuls/pull/243) ([yoheimuta](https://github.com/yoheimuta))
+- Fix changelog cache bug on Ubuntu and Debian \#235 [\#238](https://github.com/future-architect/vuls/pull/238) ([kotakanbe](https://github.com/kotakanbe))
+- add '-ssh-external' option to prepare subcommand [\#234](https://github.com/future-architect/vuls/pull/234) ([mykstmhr](https://github.com/mykstmhr))
+- Fixed error for the latest version of gocui [\#231](https://github.com/future-architect/vuls/pull/231) ([ymd38](https://github.com/ymd38))
+- Handle the refactored gocui SetCurrentView method. [\#229](https://github.com/future-architect/vuls/pull/229) ([oswell](https://github.com/oswell))
+- Fix locale env var LANG to LANGUAGE [\#215](https://github.com/future-architect/vuls/pull/215) ([kotakanbe](https://github.com/kotakanbe))
+- Fixed bug with parsing update line on CentOS/RHEL [\#206](https://github.com/future-architect/vuls/pull/206) ([andyone](https://github.com/andyone))
+- Fix defer cache.DB.close [\#201](https://github.com/future-architect/vuls/pull/201) ([kotakanbe](https://github.com/kotakanbe))
+- Fix a help message of -report-azure-blob option [\#195](https://github.com/future-architect/vuls/pull/195) ([kotakanbe](https://github.com/kotakanbe))
+- Fix error handling in tui [\#193](https://github.com/future-architect/vuls/pull/193) ([kotakanbe](https://github.com/kotakanbe))
+- Fix not working changelog cache on Container [\#189](https://github.com/future-architect/vuls/pull/189) ([kotakanbe](https://github.com/kotakanbe))
+- Fix release version detection on FreeBSD [\#184](https://github.com/future-architect/vuls/pull/184) ([kotakanbe](https://github.com/kotakanbe))
+- Fix defer cahce.DB.close\(\) [\#183](https://github.com/future-architect/vuls/pull/183) ([kotakanbe](https://github.com/kotakanbe))
+- Fix a mode of files/dir \(report, log\) [\#182](https://github.com/future-architect/vuls/pull/182) ([kotakanbe](https://github.com/kotakanbe))
+- Fix a error when no json dirs are found under results \#180 [\#181](https://github.com/future-architect/vuls/pull/181) ([kotakanbe](https://github.com/kotakanbe))
+- ssh-external option of configtest is not working \#178 [\#179](https://github.com/future-architect/vuls/pull/179) ([kotakanbe](https://github.com/kotakanbe))
+
+**Closed issues:**
+
+- Recent changes to gobui cause build failures [\#228](https://github.com/future-architect/vuls/issues/228)
+- https://hub.docker.com/r/vuls/go-cve-dictionary/ is empty [\#208](https://github.com/future-architect/vuls/issues/208)
+- Not able to install gomail fails [\#202](https://github.com/future-architect/vuls/issues/202)
+- No results file created - vuls tui failed [\#199](https://github.com/future-architect/vuls/issues/199)
+- Wrong file permissions for results/\*.json in official Docker container [\#197](https://github.com/future-architect/vuls/issues/197)
+- Failed: Unknown OS Type [\#196](https://github.com/future-architect/vuls/issues/196)
+- Segmentation fault with configtest [\#192](https://github.com/future-architect/vuls/issues/192)
+- Failed to scan. err: No server defined. Check the configuration [\#187](https://github.com/future-architect/vuls/issues/187)
+- vuls configtest -ssh-external doesnt work [\#178](https://github.com/future-architect/vuls/issues/178)
+- apt-get update: time out [\#175](https://github.com/future-architect/vuls/issues/175)
+- scanning on Centos6, but vuls recognizes debian. [\#174](https://github.com/future-architect/vuls/issues/174)
+
+**Merged pull requests:**
+
+- Update README \#225 [\#242](https://github.com/future-architect/vuls/pull/242) ([kotakanbe](https://github.com/kotakanbe))
+- fix readme [\#241](https://github.com/future-architect/vuls/pull/241) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+- Fix README \#234 [\#237](https://github.com/future-architect/vuls/pull/237) ([kotakanbe](https://github.com/kotakanbe))
+- Update glide files [\#236](https://github.com/future-architect/vuls/pull/236) ([kotakanbe](https://github.com/kotakanbe))
+- fix README [\#226](https://github.com/future-architect/vuls/pull/226) ([usiusi360](https://github.com/usiusi360))
+- fix some misspelling. [\#221](https://github.com/future-architect/vuls/pull/221) ([ymomoi](https://github.com/ymomoi))
+- fix docker readme [\#214](https://github.com/future-architect/vuls/pull/214) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+- Fix ja document about typo [\#213](https://github.com/future-architect/vuls/pull/213) ([shokohara](https://github.com/shokohara))
+- fix readme [\#212](https://github.com/future-architect/vuls/pull/212) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+- fix README [\#207](https://github.com/future-architect/vuls/pull/207) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+- fix typo [\#204](https://github.com/future-architect/vuls/pull/204) ([usiusi360](https://github.com/usiusi360))
+- fix gitignore [\#191](https://github.com/future-architect/vuls/pull/191) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+- Update glide.lock [\#188](https://github.com/future-architect/vuls/pull/188) ([kotakanbe](https://github.com/kotakanbe))
+- Fix path in setup/docker/README [\#186](https://github.com/future-architect/vuls/pull/186) ([dladuke](https://github.com/dladuke))
+- Vuls and vulsrepo are now separated [\#163](https://github.com/future-architect/vuls/pull/163) ([hikachan](https://github.com/hikachan))
+
+## [v0.1.6](https://github.com/future-architect/vuls/tree/v0.1.6) (2016-09-12)
+[Full Changelog](https://github.com/future-architect/vuls/compare/v0.1.5...v0.1.6)
+
+**Implemented enhancements:**
+
+- High speed scan on Ubuntu/Debian [\#172](https://github.com/future-architect/vuls/pull/172) ([kotakanbe](https://github.com/kotakanbe))
+- Support CWE\(Common Weakness Enumeration\) [\#169](https://github.com/future-architect/vuls/pull/169) ([kotakanbe](https://github.com/kotakanbe))
+- Enable to scan without sudo on amazon linux [\#167](https://github.com/future-architect/vuls/pull/167) ([kotakanbe](https://github.com/kotakanbe))
+- Remove deprecated options -use-unattended-upgrades,-use-yum-plugin-security [\#161](https://github.com/future-architect/vuls/pull/161) ([kotakanbe](https://github.com/kotakanbe))
+- delete sqlite3 [\#152](https://github.com/future-architect/vuls/pull/152) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+
+**Fixed bugs:**
+
+- Failed to setup vuls docker [\#170](https://github.com/future-architect/vuls/issues/170)
+- yum check-update error occurred when no reboot after kernel updating [\#165](https://github.com/future-architect/vuls/issues/165)
+- error thrown from 'docker build .' [\#157](https://github.com/future-architect/vuls/issues/157)
+- CVE-ID is truncated to 4 digits [\#153](https://github.com/future-architect/vuls/issues/153)
+- 'yum update --changelog' stalled in 'vuls scan'. if ssh user is not 'root'. [\#150](https://github.com/future-architect/vuls/issues/150)
+- Panic on packet scan [\#131](https://github.com/future-architect/vuls/issues/131)
+- Update glide.lock \#170 [\#171](https://github.com/future-architect/vuls/pull/171) ([kotakanbe](https://github.com/kotakanbe))
+- Fix detecting a platform on Azure [\#168](https://github.com/future-architect/vuls/pull/168) ([kotakanbe](https://github.com/kotakanbe))
+- Fix parse error for yum check-update \#165 [\#166](https://github.com/future-architect/vuls/pull/166) ([kotakanbe](https://github.com/kotakanbe))
+- Fix bug: Vuls on Docker [\#159](https://github.com/future-architect/vuls/pull/159) ([tjinjin](https://github.com/tjinjin))
+- Fix CVE-ID is truncated to 4 digits [\#155](https://github.com/future-architect/vuls/pull/155) ([usiusi360](https://github.com/usiusi360))
+- Fix yum update --changelog stalled when non-root ssh user on CentOS \#150 [\#151](https://github.com/future-architect/vuls/pull/151) ([kotakanbe](https://github.com/kotakanbe))
+
+**Closed issues:**
+
+- Support su for root privilege escalation [\#44](https://github.com/future-architect/vuls/issues/44)
+- Support FreeBSD [\#34](https://github.com/future-architect/vuls/issues/34)
+
+**Merged pull requests:**
+
+- Change scripts for data fetching from jvn [\#164](https://github.com/future-architect/vuls/pull/164) ([kotakanbe](https://github.com/kotakanbe))
+- Fix: setup vulsrepo [\#162](https://github.com/future-architect/vuls/pull/162) ([tjinjin](https://github.com/tjinjin))
+- Fix-docker-vulsrepo-install [\#160](https://github.com/future-architect/vuls/pull/160) ([usiusi360](https://github.com/usiusi360))
+- Reduce regular expression compilation [\#158](https://github.com/future-architect/vuls/pull/158) ([itchyny](https://github.com/itchyny))
+- Add testcases for \#153 [\#156](https://github.com/future-architect/vuls/pull/156) ([kotakanbe](https://github.com/kotakanbe))
+
+## [v0.1.5](https://github.com/future-architect/vuls/tree/v0.1.5) (2016-08-16)
+[Full Changelog](https://github.com/future-architect/vuls/compare/v0.1.4...v0.1.5)
+
+**Implemented enhancements:**
+
+- Enable to scan without running go-cve-dictionary as server mode [\#84](https://github.com/future-architect/vuls/issues/84)
+- Support high-speed scanning for CentOS [\#138](https://github.com/future-architect/vuls/pull/138) ([tai-ga](https://github.com/tai-ga))
+- Add configtest subcommand. skip un-ssh-able servers. [\#134](https://github.com/future-architect/vuls/pull/134) ([kotakanbe](https://github.com/kotakanbe))
+- Support -report-azure-blob option [\#130](https://github.com/future-architect/vuls/pull/130) ([kotakanbe](https://github.com/kotakanbe))
+- Add optional key-values that will be outputted to JSON in config [\#117](https://github.com/future-architect/vuls/pull/117) ([kotakanbe](https://github.com/kotakanbe))
+- Change dir structure [\#115](https://github.com/future-architect/vuls/pull/115) ([kotakanbe](https://github.com/kotakanbe))
+- Add some validation of loading config. user, host and port [\#113](https://github.com/future-architect/vuls/pull/113) ([kotakanbe](https://github.com/kotakanbe))
+- Support scanning with external ssh command [\#101](https://github.com/future-architect/vuls/pull/101) ([kotakanbe](https://github.com/kotakanbe))
+- Detect Platform and get instance-id of amazon ec2 [\#95](https://github.com/future-architect/vuls/pull/95) ([kotakanbe](https://github.com/kotakanbe))
+- Add -report-s3 option [\#92](https://github.com/future-architect/vuls/pull/92) ([kotakanbe](https://github.com/kotakanbe))
+- Added FreeBSD support. [\#90](https://github.com/future-architect/vuls/pull/90) ([justyntemme](https://github.com/justyntemme))
+- Add glide files for vendoring [\#89](https://github.com/future-architect/vuls/pull/89) ([kotakanbe](https://github.com/kotakanbe))
+- Fix README, change -cvedbpath to -cve-dictionary-dbpath \#84 [\#85](https://github.com/future-architect/vuls/pull/85) ([kotakanbe](https://github.com/kotakanbe))
+- Add option for it get cve detail from cve.sqlite3. [\#81](https://github.com/future-architect/vuls/pull/81) ([ymd38](https://github.com/ymd38))
+- Add -report-text option, Fix small bug of report in japanese [\#78](https://github.com/future-architect/vuls/pull/78) ([kotakanbe](https://github.com/kotakanbe))
+- Add JSONWriter, Fix CVE sort order of report [\#77](https://github.com/future-architect/vuls/pull/77) ([kotakanbe](https://github.com/kotakanbe))
+
+**Fixed bugs:**
+
+- Docker: Panic [\#76](https://github.com/future-architect/vuls/issues/76)
+- Fix apt command to scan correctly when system locale is not english [\#149](https://github.com/future-architect/vuls/pull/149) ([kit494way](https://github.com/kit494way))
+- Disable -ask-sudo-password for security reasons [\#148](https://github.com/future-architect/vuls/pull/148) ([kotakanbe](https://github.com/kotakanbe))
+- Fix no tty error while executing with -external-ssh option [\#143](https://github.com/future-architect/vuls/pull/143) ([kotakanbe](https://github.com/kotakanbe))
+- wrong log packages [\#141](https://github.com/future-architect/vuls/pull/141) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+- Fix platform detection. [\#137](https://github.com/future-architect/vuls/pull/137) ([Rompei](https://github.com/Rompei))
+- Fix nil pointer when scan with -cve-dictionary-dbpath and cpeNames [\#111](https://github.com/future-architect/vuls/pull/111) ([kotakanbe](https://github.com/kotakanbe))
+- Remove vulndb file before pkg audit [\#110](https://github.com/future-architect/vuls/pull/110) ([kotakanbe](https://github.com/kotakanbe))
+- Add error handling when unable to connect via ssh. status code: 255 [\#108](https://github.com/future-architect/vuls/pull/108) ([kotakanbe](https://github.com/kotakanbe))
+- Enable to detect vulnerabilities on FreeBSD [\#98](https://github.com/future-architect/vuls/pull/98) ([kotakanbe](https://github.com/kotakanbe))
+- Fix unknown format err while check-update on RHEL6.5 [\#93](https://github.com/future-architect/vuls/pull/93) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+- Fix type of SMTP Port of discovery command's output [\#88](https://github.com/future-architect/vuls/pull/88) ([kotakanbe](https://github.com/kotakanbe))
+- Fix error msg when go-cve-dictionary is unavailable \#84 [\#86](https://github.com/future-architect/vuls/pull/86) ([kotakanbe](https://github.com/kotakanbe))
+- Fix error handling to avoid nil pointer err on debian [\#83](https://github.com/future-architect/vuls/pull/83) ([kotakanbe](https://github.com/kotakanbe))
+- Fix nil pointer while doing apt-cache policy on ubuntu \#76 [\#82](https://github.com/future-architect/vuls/pull/82) ([kotakanbe](https://github.com/kotakanbe))
+- fix log import url [\#79](https://github.com/future-architect/vuls/pull/79) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+- Fix error handling of gorequest [\#75](https://github.com/future-architect/vuls/pull/75) ([kotakanbe](https://github.com/kotakanbe))
+- Fix freezing forever when no args specified in TUI mode [\#73](https://github.com/future-architect/vuls/pull/73) ([kotakanbe](https://github.com/kotakanbe))
+- mv version.go version/version.go to run main.go without compile [\#71](https://github.com/future-architect/vuls/pull/71) ([sadayuki-matsuno](https://github.com/sadayuki-matsuno))
+
+**Closed issues:**
+
+- SSh password authentication failed on FreeBSD [\#99](https://github.com/future-architect/vuls/issues/99)
+- BUG: -o pipefail is not work on FreeBSD's /bin/sh. because it isn't bash [\#91](https://github.com/future-architect/vuls/issues/91)
+- Use ~/.ssh/config [\#62](https://github.com/future-architect/vuls/issues/62)
+- SSH ciphers [\#37](https://github.com/future-architect/vuls/issues/37)
+
+**Merged pull requests:**
+
+- Update README \#138 [\#144](https://github.com/future-architect/vuls/pull/144) ([kotakanbe](https://github.com/kotakanbe))
+- Fix a typo [\#142](https://github.com/future-architect/vuls/pull/142) ([dtan4](https://github.com/dtan4))
+- Remove unnecessary step in readme of docker setup [\#140](https://github.com/future-architect/vuls/pull/140) ([mikkame](https://github.com/mikkame))
+- Update logo [\#139](https://github.com/future-architect/vuls/pull/139) ([chanomaru](https://github.com/chanomaru))
+- Update README.ja.md to fix wrong tips. [\#135](https://github.com/future-architect/vuls/pull/135) ([a2atsu](https://github.com/a2atsu))
+- add tips about NVD JVN issue [\#133](https://github.com/future-architect/vuls/pull/133) ([a2atsu](https://github.com/a2atsu))
+- Fix README wrong links [\#129](https://github.com/future-architect/vuls/pull/129) ([aomoriringo](https://github.com/aomoriringo))
+- Add logo [\#126](https://github.com/future-architect/vuls/pull/126) ([chanomaru](https://github.com/chanomaru))
+- Improve setup/docker [\#125](https://github.com/future-architect/vuls/pull/125) ([kotakanbe](https://github.com/kotakanbe))
+- Fix scan command help [\#124](https://github.com/future-architect/vuls/pull/124) ([aomoriringo](https://github.com/aomoriringo))
+- added dockernized-vuls with vulsrepo [\#121](https://github.com/future-architect/vuls/pull/121) ([hikachan](https://github.com/hikachan))
+- Fix detect platform on azure and degital ocean [\#119](https://github.com/future-architect/vuls/pull/119) ([kotakanbe](https://github.com/kotakanbe))
+- Remove json marshall-indent [\#118](https://github.com/future-architect/vuls/pull/118) ([kotakanbe](https://github.com/kotakanbe))
+- Improve Readme.ja [\#116](https://github.com/future-architect/vuls/pull/116) ([kotakanbe](https://github.com/kotakanbe))
+- Add architecture diag to README.md [\#114](https://github.com/future-architect/vuls/pull/114) ([kotakanbe](https://github.com/kotakanbe))
+- Rename linux.go to base.go [\#100](https://github.com/future-architect/vuls/pull/100) ([kotakanbe](https://github.com/kotakanbe))
+- Update README.md [\#74](https://github.com/future-architect/vuls/pull/74) ([yoshi-taka](https://github.com/yoshi-taka))
+- Refactoring debian.go [\#72](https://github.com/future-architect/vuls/pull/72) ([kotakanbe](https://github.com/kotakanbe))
+
+## [v0.1.4](https://github.com/future-architect/vuls/tree/v0.1.4) (2016-05-24)
+[Full Changelog](https://github.com/future-architect/vuls/compare/v0.1.3...v0.1.4)
+
+**Implemented enhancements:**
+
+- Initial fetch from NVD is too heavy \(2.3 GB of memory consumed\) [\#27](https://github.com/future-architect/vuls/issues/27)
+- Enable to show previous scan result [\#69](https://github.com/future-architect/vuls/pull/69) ([kotakanbe](https://github.com/kotakanbe))
+- Add ignore-unscored-cves option [\#68](https://github.com/future-architect/vuls/pull/68) ([kotakanbe](https://github.com/kotakanbe))
+- Support dynamic scanning docker container [\#67](https://github.com/future-architect/vuls/pull/67) ([kotakanbe](https://github.com/kotakanbe))
+- Add version flag [\#65](https://github.com/future-architect/vuls/pull/65) ([kotakanbe](https://github.com/kotakanbe))
+- Update Dockerfile [\#57](https://github.com/future-architect/vuls/pull/57) ([theonlydoo](https://github.com/theonlydoo))
+- Update run.sh [\#56](https://github.com/future-architect/vuls/pull/56) ([theonlydoo](https://github.com/theonlydoo))
+- Support Windows [\#33](https://github.com/future-architect/vuls/pull/33) ([mattn](https://github.com/mattn))
+
+**Fixed bugs:**
+
+- vuls scan -cvss-over does not work. [\#59](https://github.com/future-architect/vuls/issues/59)
+- `panic: runtime error: invalid memory address or nil pointer dereference` when scan CentOS5.5 [\#58](https://github.com/future-architect/vuls/issues/58)
+-  It rans out of memory. [\#47](https://github.com/future-architect/vuls/issues/47)
+- BUG: vuls scan on CentOS with Japanese environment. [\#43](https://github.com/future-architect/vuls/issues/43)
+- yum --color=never [\#36](https://github.com/future-architect/vuls/issues/36)
+- Failed to parse yum check-update [\#32](https://github.com/future-architect/vuls/issues/32)
+- Pointless sudo [\#29](https://github.com/future-architect/vuls/issues/29)
+- Can't init database in a path having blanks [\#26](https://github.com/future-architect/vuls/issues/26)
+- Fix pointless sudo in debian.go \#29 [\#66](https://github.com/future-architect/vuls/pull/66) ([kotakanbe](https://github.com/kotakanbe))
+- Fix error handling of httpGet in cve-client \#58 [\#64](https://github.com/future-architect/vuls/pull/64) ([kotakanbe](https://github.com/kotakanbe))
+- Fix nil pointer at error handling of cve\_client \#58 [\#63](https://github.com/future-architect/vuls/pull/63) ([kotakanbe](https://github.com/kotakanbe))
+- Set language en\_US. [\#61](https://github.com/future-architect/vuls/pull/61) ([pabroff](https://github.com/pabroff))
+- Fix -cvss-over flag \#59 [\#60](https://github.com/future-architect/vuls/pull/60) ([kotakanbe](https://github.com/kotakanbe))
+- Fix scan on Japanese environment. [\#55](https://github.com/future-architect/vuls/pull/55) ([pabroff](https://github.com/pabroff))
+- Fix a typo: replace Depricated by Deprecated. [\#54](https://github.com/future-architect/vuls/pull/54) ([jody-frankowski](https://github.com/jody-frankowski))
+- Fix yes no infinite loop while doing yum update --changelog on root@CentOS \#47 [\#50](https://github.com/future-architect/vuls/pull/50) ([pabroff](https://github.com/pabroff))
+- Fix $servername in output of discover command [\#45](https://github.com/future-architect/vuls/pull/45) ([kotakanbe](https://github.com/kotakanbe))
+
+## [v0.1.3](https://github.com/future-architect/vuls/tree/v0.1.3) (2016-04-21)
+[Full Changelog](https://github.com/future-architect/vuls/compare/v0.1.2...v0.1.3)
+
+**Implemented enhancements:**
+
+- Add sudo support for prepare [\#11](https://github.com/future-architect/vuls/issues/11)
+- Dockerfile? [\#10](https://github.com/future-architect/vuls/issues/10)
+- Update README [\#41](https://github.com/future-architect/vuls/pull/41) ([theonlydoo](https://github.com/theonlydoo))
+- Sparse dockerization [\#38](https://github.com/future-architect/vuls/pull/38) ([theonlydoo](https://github.com/theonlydoo))
+- No password in config [\#35](https://github.com/future-architect/vuls/pull/35) ([kotakanbe](https://github.com/kotakanbe))
+- Fr readme translation [\#23](https://github.com/future-architect/vuls/pull/23) ([novakin](https://github.com/novakin))
+
+**Fixed bugs:**
+
+- Issues updating CVE database behind https proxy [\#39](https://github.com/future-architect/vuls/issues/39)
+- Vuls failed to parse yum check-update [\#24](https://github.com/future-architect/vuls/issues/24)
+- Fix yum to yum --color=never \#36 [\#42](https://github.com/future-architect/vuls/pull/42) ([kotakanbe](https://github.com/kotakanbe))
+- Fix parse yum check update [\#40](https://github.com/future-architect/vuls/pull/40) ([kotakanbe](https://github.com/kotakanbe))
+- fix typo [\#31](https://github.com/future-architect/vuls/pull/31) ([blue119](https://github.com/blue119))
+- Fix error while parsing yum check-update \#24 [\#30](https://github.com/future-architect/vuls/pull/30) ([kotakanbe](https://github.com/kotakanbe))
+
+**Closed issues:**
+
+- Unable to scan on ubuntu because changelog.ubuntu.com is down... [\#21](https://github.com/future-architect/vuls/issues/21)
+- err: Not initialize\(d\) yet.. [\#16](https://github.com/future-architect/vuls/issues/16)
+- Errors when using fish shell [\#8](https://github.com/future-architect/vuls/issues/8)
+
 ## [v0.1.2](https://github.com/future-architect/vuls/tree/v0.1.2) (2016-04-12)
 [Full Changelog](https://github.com/future-architect/vuls/compare/v0.1.1...v0.1.2)
 

ISSUE_TEMPLATE

@@ -0,0 +1,36 @@
+
+# Environment
+
+## Vuls
+
+Hash : ____
+
+To check the commit hash of HEAD
+$ vuls -v
+
+or
+$ cd $GOPATH/src/github.com/future-architect/vuls 
+$ git rev-parse --short HEAD 
+
+## OS
+- Target Server: Write here
+- Vuls Server: Write here
+
+## Go
+- Go version: here
+
+# Current Output
+
+Please re-run the command using ```-debug``` and provide the output below.
+
+# Addition Details
+
+Can you also please fill in each of the remaining sections.
+
+## Expected Behavior
+
+## Actual Behavior
+
+## Steps to reproduce the behaviour
+
+

Makefile

@@ -1,4 +1,9 @@
 .PHONY: \
+	glide \
+	deps \
+	update \
+	build \
+	install \
 	all \
 	vendor \
 	lint \
@@ -7,25 +12,39 @@
 	fmtcheck \
 	pretest \
 	test \
-	integration \
 	cov \
 	clean
 
 SRCS = $(shell git ls-files '*.go')
-PKGS = ./. ./db ./config ./models ./report ./cveapi ./scan ./util ./commands
+PKGS = ./. ./config ./models ./report ./cveapi ./scan ./util ./commands ./cache
+VERSION := $(shell git describe --tags --abbrev=0)
+REVISION := $(shell git rev-parse --short HEAD)
+LDFLAGS := -X 'main.version=$(VERSION)' \
+	-X 'main.revision=$(REVISION)'
+
+glide:
+	go get github.com/Masterminds/glide
+
+deps: glide
+	glide install
+
+update: glide
+	glide update
+
+build: main.go deps
+	go build -ldflags "$(LDFLAGS)" -o vuls $<
+
+install: main.go deps
+	go install -ldflags "$(LDFLAGS)"
 
 all: test
 
-vendor:
-	@ go get -v github.com/mjibson/party
-	party -d external -c -u
-
 lint:
 	@ go get -v github.com/golang/lint/golint
 	$(foreach file,$(SRCS),golint $(file) || exit;)
 
 vet:
-	@-go get -v golang.org/x/tools/cmd/vet
+	#  @-go get -v golang.org/x/tools/cmd/vet
 	$(foreach pkg,$(PKGS),go vet $(pkg);)
 
 fmt:

README.fr.md

@@ -7,6 +7,7 @@ Scanneur de vulnérabilité Linux, sans agent, écrit en golang
 
 Nous avons une équipe Slack. [Rejoignez notre Slack Team](http://goo.gl/forms/xm5KFo35tu)  
 
+[README en English](https://github.com/future-architect/vuls/blob/master/README.md)  
 [README en Japonais](https://github.com/future-architect/vuls/blob/master/README.ja.md)  
 
 [![asciicast](https://asciinema.org/a/3y9zrf950agiko7klg8abvyck.png)](https://asciinema.org/a/3y9zrf950agiko7klg8abvyck)
@@ -106,14 +107,14 @@ Vuls requiert l'installation des paquets suivants :
 - sqlite
 - git
 - gcc
-- go v1.6
+- go v1.7.1 or later
     - https://golang.org/doc/install
 
 ```bash
 $ ssh ec2-user@52.100.100.100  -i ~/.ssh/private.pem
 $ sudo yum -y install sqlite git gcc
-$ wget https://storage.googleapis.com/golang/go1.6.linux-amd64.tar.gz
-$ sudo tar -C /usr/local -xzf go1.6.linux-amd64.tar.gz
+$ wget https://storage.googleapis.com/golang/go1.7.1.linux-amd64.tar.gz
+$ sudo tar -C /usr/local -xzf go1.7.1.linux-amd64.tar.gz
 $ mkdir $HOME/go
 ```
 Ajoutez les lignes suivantes dans /etc/profile.d/goenv.sh
@@ -144,22 +145,6 @@ Démarrez go-cve-dictionary en mode serveur.
 Lors de son premier démarrage go-cve-dictionary récupère la liste des vulnérabilités depuis NVD
 Cette opération prend environ 10 minutes (sur AWS).  
 
-```bash
-$ go-cve-dictionary server
-... Fetching ...
-$ ls -alh cve.sqlite3
--rw-r--r-- 1 ec2-user ec2-user 7.0M Mar 24 13:20 cve.sqlite3
-```
-
-Une fois les informations de vulnérabilités collectées redémarrez le mode serveur.
-```bash
-$ go-cve-dictionary server
-[Mar 24 15:21:55]  INFO Opening DB. datafile: /home/ec2-user/cve.sqlite3
-[Mar 24 15:21:55]  INFO Migrating DB
-[Mar 24 15:21:56]  INFO Starting HTTP Sever...
-[Mar 24 15:21:56]  INFO Listening on 127.0.0.1:1323
-```
-
 ## Step5. Déploiement de Vuls
 
 Ouvrez un second terminal, connectez vous à l'instance ec2 via SSH
@@ -193,7 +178,7 @@ $ vuls prepare
 ## Step8. Scan
 
 ```
-$ vuls scan
+$ vuls scan -cve-dictionary-dbpath=$PWD/cve.sqlite3
 INFO[0000] Begin scanning (config: /home/ec2-user/config.toml)
 
 ... snip ...
@@ -216,7 +201,7 @@ Summary         Unspecified vulnerability in the Java SE and Java SE Embedded co
 NVD             https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2016-0494
 MITRE           https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-0494
 CVE Details     http://www.cvedetails.com/cve/CVE-2016-0494
-CVSS Claculator https://nvd.nist.gov/cvss/v2-calculator?name=CVE-2016-0494&vector=(AV:N/AC:L/Au:N/C:C/I:C/A:C)
+CVSS Calculator https://nvd.nist.gov/cvss/v2-calculator?name=CVE-2016-0494&vector=(AV:N/AC:L/Au:N/C:C/I:C/A:C)
 RHEL-CVE        https://access.redhat.com/security/cve/CVE-2016-0494
 ALAS-2016-643   https://alas.aws.amazon.com/ALAS-2016-643.html
 Package/CPE     java-1.7.0-openjdk-1.7.0.91-2.6.2.2.63.amzn1 -> java-1.7.0-openjdk-1:1.7.0.95-2.6.4.0.65.amzn1
@@ -236,417 +221,4 @@ $ vuls tui
 
 ----
 
-# Architecture
-
-![Vuls-Architecture](img/vuls-architecture.png)
-
-## go-cve-dictinary  
-- Collecte les informations de vulnérabilités depuis NVD, JVN(Japonais), et les envoie dans SQLite.
-
-## Vuls
-- Scan de vulnérabilités sur serveurs et création d'une liste contenant les CVE ID
-- Pour des informations plus détaillés sur une CVE, envoie une requete HTTP à go-cve-dictinary 
-- Rapport à Slack et par Email
-- L'administrateur système peut voir les résultats du dernier rapport dans le terminal
-
-----
-
-# Exemples d'utilisation
-
-## Scan de tous les serverus
-
-![Vuls-Usecase1](img/vuls-usecase-elb-rails-rds-all.png)
-
-## Scan d'un seul serveur
-
-web/app server in the same configuration under the load balancer
-
-![Vuls-Usecase2](img/vuls-usecase-elb-rails-rds-single.png)
-
-----
-
-# OS supportés 
-
-| Distribution|            Release |
-|:------------|-------------------:|
-| Ubuntu      |          12, 14, 16|
-| Debian      |                7, 8|
-| RHEL        |          4, 5, 6, 7|
-| CentOS      |             5, 6, 7|
-| Amazon Linux|                All |
-
-----
-
-
-# Usage: Détection Automatique de Serveurs 
-
-La sous-commande Discovery permet de détecter les serveurs actifs dans un range d'IP CIDR, les résultas sont directement affichés dans le terminal en respectant le format du fichier de configuration (TOML format).
-
-```
-$ vuls discover -help
-discover:
-        discover 192.168.0.0/24
-```
-
-## Exemple
-
-```
-$ vuls discover 172.31.4.0/24
-# Create config.toml using below and then ./vuls --config=/path/to/config.toml
-
-[slack]
-hookURL      = "https://hooks.slack.com/services/abc123/defghijklmnopqrstuvwxyz"
-channel      = "#channel-name"
-#channel      = "#{servername}"
-iconEmoji    = ":ghost:"
-authUser     = "username"
-notifyUsers  = ["@username"]
-
-[mail]
-smtpAddr      = "smtp.gmail.com"
-smtpPort      = 465
-user          = "username"
-password      = "password"
-from          = "from@address.com"
-to            = ["to@address.com"]
-cc            = ["cc@address.com"]
-subjectPrefix = "[vuls]"
-
-[default]
-#port        = "22"
-#user        = "username"
-#password    = "password"
-#keyPath     = "/home/username/.ssh/id_rsa"
-#keyPassword = "password"
-
-[servers]
-
-[servers.172-31-4-82]
-host         = "172.31.4.82"
-#port        = "22"
-#user        = "root"
-#password    = "password"
-#keyPath     = "/home/username/.ssh/id_rsa"
-#keyPassword = "password"
-#cpeNames = [
-#  "cpe:/a:rubyonrails:ruby_on_rails:4.2.1",
-#]
-```
-
-Vous pouvez customiser votre configuration en utilisant ce modèle.
-
-----
-
-# Configuration
-
-- Slack section
-    ```
-    [slack]
-    hookURL      = "https://hooks.slack.com/services/abc123/defghijklmnopqrstuvwxyz"
-    channel      = "#channel-name"
-    #channel      = "#{servername}"
-    iconEmoji    = ":ghost:"
-    authUser     = "username"
-    notifyUsers  = ["@username"]
-    ```
-
-    - hookURL : Incomming webhook's URL  
-    - channel : channel name.  
-    If you set #{servername} to channel, the report will be sent to #servername channel.  
-    In the following example, the report will be sent to the #server1 and #server2.  
-    Be sure to create these channels before scanning.
-      ```
-      [slack]
-      channel      = "#{servername}"
-      ...snip...
-
-      [servers]
-
-      [servers.server1]
-      host         = "172.31.4.82"
-      ...snip...
-
-      [servers.server2]
-      host         = "172.31.4.83"
-      ...snip...
-      ```
-
-    - iconEmoji: emoji
-    - authUser: username of the slack team
-    - notifyUsers: a list of Slack usernames to send Slack notifications.
-      If you set ["@foo", "@bar"] to notifyUsers, @foo @bar will be included in text.  
-      So @foo, @bar can receive mobile push notifications on their smartphone.  
-
-- Mail section
-    ```
-    [mail]
-    smtpAddr      = "smtp.gmail.com"
-    smtpPort      = 465
-    user          = "username"
-    password      = "password"
-    from          = "from@address.com"
-    to            = ["to@address.com"]
-    cc            = ["cc@address.com"]
-    subjectPrefix = "[vuls]"
-    ```
-
-- Default section
-    ```
-    [default]
-    #port        = "22"
-    #user        = "username"
-    #password    = "password"
-    #keyPath     = "/home/username/.ssh/id_rsa"
-    #keyPassword = "password"
-    ```
-    Items of the default section will be used if not specified.
-
-- servers section
-    ```
-    [servers]
-
-    [servers.172-31-4-82]
-    host         = "172.31.4.82"
-    #port        = "22"
-    #user        = "root"
-    #password    = "password"
-    #keyPath     = "/home/username/.ssh/id_rsa"
-    #keyPassword = "password"
-    #cpeNames = [
-    #  "cpe:/a:rubyonrails:ruby_on_rails:4.2.1",
-    #]
-    ```
-    Vous pouvez remplacer les valeurs par défaut indiquées en modifiant la section default
-    Vuls supporte plusieurs méthodes d'authentification SSH :
-    - SSH agent
-    - SSH authentication par clés (avec mot de passe ou sans mot de passe)
-    - Authentification par mot de passe
-
-----
-
-# Utilisation : Prepare
-
-La sous-commande prepare installe tous les paquets nécessaires sur chaque serveur.
-
-| Distribution|            Release | Requirements |
-|:------------|-------------------:|:-------------|
-| Ubuntu      |          12, 14, 16| -            |
-| Debian      |                7, 8| apptitude    |
-| CentOS      |                   5| yum-plugin-security, yum-changelog |
-| CentOS      |                6, 7| yum-plugin-security, yum-plugin-changelog |
-| Amazon      |                All | -            |
-| RHEL        |         4, 5, 6, 7 | -            |
-
-
-```
-$ vuls prepare -help
-prepare:
-        prepare [-config=/path/to/config.toml] [-debug]
-
-  -config string
-        /path/to/toml (default "$PWD/config.toml")
-  -debug
-        debug mode
-  -use-unattended-upgrades
-        [Deprecated] For Ubuntu, install unattended-upgrades
-```
-
-----
-
-# Utilisation : Scan
-
-```
-$ vuls scan -help
-scan:
-        scan
-                [-lang=en|ja]
-                [-config=/path/to/config.toml]
-                [-dbpath=/path/to/vuls.sqlite3]
-                [-cve-dictionary-url=http://127.0.0.1:1323]
-                [-cvss-over=7]
-                [-report-slack]
-                [-report-mail]
-                [-http-proxy=http://192.168.0.1:8080]
-                [-debug]
-                [-debug-sql]
-  -config string
-        /path/to/toml (default "$PWD/config.toml")
-  -cve-dictionary-url string
-        http://CVE.Dictionary (default "http://127.0.0.1:1323")
-  -cvss-over float
-        -cvss-over=6.5 means reporting CVSS Score 6.5 and over (default: 0 (means report all))
-  -dbpath string
-        /path/to/sqlite3 (default "$PWD/vuls.sqlite3")
-  -debug
-        debug mode
-  -debug-sql
-        SQL debug mode
-  -http-proxy string
-        http://proxy-url:port (default: empty)
-  -lang string
-        [en|ja] (default "en")
-  -report-mail
-        Email report
-  -report-slack
-        Slack report
-  -use-unattended-upgrades
-        [Deprecated] For Ubuntu. Scan by unattended-upgrades or not (use apt-get upgrade --dry-run by default)
-  -use-yum-plugin-security
-        [Deprecated] For CentOS 5. Scan by yum-plugin-security or not (use yum check-update by default)
-
-```
-
-## exemple
-
-Lancez go-cve-dictionary en mode serveur avant de lancer un scan
-```
-$ go-cve-dictionary server
-```
-
-### Scan tous les serveurs identifiés dans le fichier de configuration
-```
-$ vuls scan --report-slack --report-mail --cvss-over=7
-```
-Via cette simple commande Vuls va : ..
-- Scanner tous les serveurs identifiés dans le fichier de configuration
-- Envoyer les résultas du scan à slack et par email
-- Ne rapporter que les CVE dont la note CVSS est au dessus de 7
-- Afficher les résultats du scan dans le terminal
-
-### Scan de serveurs spécifiques
-```
-$ vuls scan server1 server2
-```
-Via cette simple commande Vuls va : ..
-- Scanner seulement 2 serveurs. (server1, server2)
-- Afficher les résultats du scan dans le terminal
-
-----
-
-# Utilisation : Recherche de vulnérabilités sur des paquets non compris dans l'OS
-
-Il est possible de détecter des vulnérabilités sur des programmes que vous avez compilés, des lors que les libraries et frameworks ont été enregistré dans [CPE](https://nvd.nist.gov/cpe.cfm).
-
--  Comment rechercher dans CPE via le nom du programme
-    - [NVD: Search Common Platform Enumerations (CPE)](https://web.nvd.nist.gov/view/cpe/search)  
-    **Check CPE Naming Format: 2.2**
-
-- Configuration  
-Pour détecter des vulnérabilités sur Ruby on Rails v4.2.1, cpeNames doit etre déclaré dans la section servers.
-    ```
-    [servers]
-
-    [servers.172-31-4-82]
-    host         = "172.31.4.82"
-    user        = "ec2-user"
-    keyPath     = "/home/username/.ssh/id_rsa"
-    cpeNames = [
-      "cpe:/a:rubyonrails:ruby_on_rails:4.2.1",
-    ]
-    ```
-
-# Utilisation : Mise à jour des données NVD.
-
-```
-$ go-cve-dictionary fetchnvd -h
-fetchnvd:
-        fetchnvd
-                [-last2y]
-                [-dbpath=/path/to/cve.sqlite3]
-                [-debug]
-                [-debug-sql]
-
-  -dbpath string
-        /path/to/sqlite3 (default "$PWD/cve.sqlite3")
-  -debug
-        debug mode
-  -debug-sql
-        SQL debug mode
-  -last2y
-        Refresh NVD data in the last two years.
-```
-
-- Récupérer toutes les données jusqu'à aujourd'hui
-
-```
-$ go-cve-dictionary fetchnvd -entire
-```
-
-- Reçupérer les données des 2 denières années
-
-```
-$ go-cve-dictionary fetchnvd -last2y
-```
-
-----
-
-# Misc
-
-- HTTP Proxy Support  
-If your system is behind HTTP proxy, you have to specify --http-proxy option.
-
-- How to Daemonize go-cve-dictionary  
-Use Systemd, Upstart or supervisord, daemontools...
-
-- How to Enable Automatic-Update of Vunerability Data.  
-Use job scheduler like Cron (with -last2y option).
-
-- How to cross compile
-    ```bash
-    $ cd /path/to/your/local-git-reporsitory/vuls
-    $ GOOS=linux GOARCH=amd64 go build -o vuls.amd64
-    ```
-
-- Logging  
-Log wrote to under /var/log/vuls/
-
-- Debug  
-Run with --debug, --sql-debug option.
-
-- Ajusting Open File Limit  
-[Riak docs](http://docs.basho.com/riak/latest/ops/tuning/open-files-limit/) is awesome.
-
-- Does Vuls accept ssh connections with fish-shell or old zsh as the login shell?  
-No, Vuls needs a user on the server for bash login. see also [#8](/../../issues/8)
-
-- Windows  
-Use Microsoft Baseline Security Analyzer. [MBSA](https://technet.microsoft.com/en-us/security/cc184924.aspx)
-
-----
-
-# Data Source
-
-- [NVD](https://nvd.nist.gov/)
-- [JVN(Japanese)](http://jvndb.jvn.jp/apis/myjvn/)
-
-
-# Authors
-
-kotakanbe ([@kotakanbe](https://twitter.com/kotakanbe)) created vuls and [these fine people](https://github.com/future-architect/vuls/graphs/contributors) have contributed.
-
-----
-
-# Contribute
-
-1. Fork it
-2. Create your feature branch (`git checkout -b my-new-feature`)
-3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Push to the branch (`git push origin my-new-feature`)
-5. Create new Pull Request
-
-----
-
-# Change Log
-
-Please see [CHANGELOG](https://github.com/future-architect/vuls/blob/master/CHANGELOG.md).
-
-----
-
-# Licence
-
-Please see [LICENSE](https://github.com/future-architect/vuls/blob/master/LICENSE).
-
-
-[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/future-architect/vuls/trend.png)](https://bitdeli.com/free "Bitdeli Badge")
-
-
+For more information see [README in English](https://github.com/future-architect/vuls/blob/master/README.md)  

cache/bolt.go

@@ -0,0 +1,173 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package cache
+
+import (
+	"encoding/json"
+	"fmt"
+
+	"github.com/Sirupsen/logrus"
+	"github.com/boltdb/bolt"
+	"github.com/future-architect/vuls/util"
+)
+
+// Bolt holds a pointer of bolt.DB
+// boltdb is used to store a cache of Changelogs of Ubuntu/Debian
+type Bolt struct {
+	Path string
+	Log  *logrus.Entry
+	db   *bolt.DB
+}
+
+// SetupBolt opens a boltdb and creates a meta bucket if not exists.
+func SetupBolt(path string, l *logrus.Entry) error {
+	l.Infof("Open boltDB: %s", path)
+	db, err := bolt.Open(path, 0600, nil)
+	if err != nil {
+		return err
+	}
+
+	b := Bolt{
+		Path: path,
+		Log:  l,
+		db:   db,
+	}
+	if err = b.createBucketIfNotExists(metabucket); err != nil {
+		return err
+	}
+
+	DB = b
+	return nil
+}
+
+// Close a db.
+func (b Bolt) Close() error {
+	if b.db == nil {
+		return nil
+	}
+	return b.db.Close()
+}
+
+//  CreateBucketIfNotExists creates a buket that is specified by arg.
+func (b *Bolt) createBucketIfNotExists(name string) error {
+	return b.db.Update(func(tx *bolt.Tx) error {
+		_, err := tx.CreateBucketIfNotExists([]byte(name))
+		if err != nil {
+			return fmt.Errorf("Failed to create bucket: %s", err)
+		}
+		return nil
+	})
+}
+
+// GetMeta gets a Meta Information os the servername to boltdb.
+func (b Bolt) GetMeta(serverName string) (meta Meta, found bool, err error) {
+	err = b.db.View(func(tx *bolt.Tx) error {
+		bkt := tx.Bucket([]byte(metabucket))
+		v := bkt.Get([]byte(serverName))
+		if len(v) == 0 {
+			found = false
+			return nil
+		}
+		if e := json.Unmarshal(v, &meta); e != nil {
+			return e
+		}
+		found = true
+		return nil
+	})
+	return
+}
+
+// EnsureBuckets puts a Meta information and create a buket that holds changelogs.
+func (b Bolt) EnsureBuckets(meta Meta) error {
+	jsonBytes, err := json.Marshal(meta)
+	if err != nil {
+		return fmt.Errorf("Failed to marshal to JSON: %s", err)
+	}
+	return b.db.Update(func(tx *bolt.Tx) error {
+		b.Log.Debugf("Put to meta: %s", meta.Name)
+		bkt := tx.Bucket([]byte(metabucket))
+		if err := bkt.Put([]byte(meta.Name), jsonBytes); err != nil {
+			return err
+		}
+
+		// re-create a bucket (bucket name: servername)
+		bkt = tx.Bucket([]byte(meta.Name))
+		if bkt != nil {
+			b.Log.Debugf("Delete bucket: %s", meta.Name)
+			if err := tx.DeleteBucket([]byte(meta.Name)); err != nil {
+				return err
+			}
+			b.Log.Debugf("Bucket deleted: %s", meta.Name)
+		}
+		b.Log.Debugf("Create bucket: %s", meta.Name)
+		if _, err := tx.CreateBucket([]byte(meta.Name)); err != nil {
+			return err
+		}
+		b.Log.Debugf("Bucket created: %s", meta.Name)
+		return nil
+	})
+}
+
+// PrettyPrint is for debuging
+func (b Bolt) PrettyPrint(meta Meta) error {
+	return b.db.View(func(tx *bolt.Tx) error {
+		bkt := tx.Bucket([]byte(metabucket))
+		v := bkt.Get([]byte(meta.Name))
+		b.Log.Debugf("key:%s, value:%s", meta.Name, v)
+
+		bkt = tx.Bucket([]byte(meta.Name))
+		c := bkt.Cursor()
+		for k, v := c.First(); k != nil; k, v = c.Next() {
+			b.Log.Debugf("key:%s, len: %d, %s...",
+				k, len(v), util.Truncate(string(v), 30))
+		}
+		return nil
+	})
+}
+
+// GetChangelog get the changelgo of specified packName from the Bucket
+func (b Bolt) GetChangelog(servername, packName string) (changelog string, err error) {
+	err = b.db.View(func(tx *bolt.Tx) error {
+		bkt := tx.Bucket([]byte(servername))
+		if bkt == nil {
+			return fmt.Errorf("Faild to get Bucket: %s", servername)
+		}
+		v := bkt.Get([]byte(packName))
+		if v == nil {
+			changelog = ""
+			return nil
+		}
+		changelog = string(v)
+		return nil
+	})
+	return
+}
+
+// PutChangelog put the changelgo of specified packName into the Bucket
+func (b Bolt) PutChangelog(servername, packName, changelog string) error {
+	return b.db.Update(func(tx *bolt.Tx) error {
+		bkt := tx.Bucket([]byte(servername))
+		if bkt == nil {
+			return fmt.Errorf("Faild to get Bucket: %s", servername)
+		}
+		if err := bkt.Put([]byte(packName), []byte(changelog)); err != nil {
+			return err
+		}
+		return nil
+	})
+}

cache/bolt_test.go

@@ -0,0 +1,134 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package cache
+
+import (
+	"os"
+	"reflect"
+	"testing"
+
+	"github.com/Sirupsen/logrus"
+	"github.com/boltdb/bolt"
+	"github.com/future-architect/vuls/config"
+	"github.com/future-architect/vuls/models"
+)
+
+const path = "/tmp/vuls-test-cache-11111111.db"
+const servername = "server1"
+
+var meta = Meta{
+	Name: servername,
+	Distro: config.Distro{
+		Family:  "ubuntu",
+		Release: "16.04",
+	},
+	Packs: []models.PackageInfo{
+		{
+			Name:    "apt",
+			Version: "1",
+		},
+	},
+}
+
+func TestSetupBolt(t *testing.T) {
+	log := logrus.NewEntry(&logrus.Logger{})
+	err := SetupBolt(path, log)
+	if err != nil {
+		t.Errorf("Failed to setup bolt: %s", err)
+	}
+	defer os.Remove(path)
+
+	if err := DB.Close(); err != nil {
+		t.Errorf("Failed to close bolt: %s", err)
+	}
+
+	// check if meta bucket exists
+	db, err := bolt.Open(path, 0600, nil)
+	if err != nil {
+		t.Errorf("Failed to open bolt: %s", err)
+	}
+
+	db.View(func(tx *bolt.Tx) error {
+		bkt := tx.Bucket([]byte(metabucket))
+		if bkt == nil {
+			t.Errorf("Meta bucket nof found")
+		}
+		return nil
+	})
+
+}
+
+func TestEnsureBuckets(t *testing.T) {
+	log := logrus.NewEntry(&logrus.Logger{})
+	if err := SetupBolt(path, log); err != nil {
+		t.Errorf("Failed to setup bolt: %s", err)
+	}
+	if err := DB.EnsureBuckets(meta); err != nil {
+		t.Errorf("Failed to ensure buckets: %s", err)
+	}
+	defer os.Remove(path)
+
+	m, found, err := DB.GetMeta(servername)
+	if err != nil {
+		t.Errorf("Failed to get meta: %s", err)
+	}
+	if !found {
+		t.Errorf("Not Found in meta")
+	}
+	if !reflect.DeepEqual(meta, m) {
+		t.Errorf("expected %v, actual %v", meta, m)
+	}
+	if err := DB.Close(); err != nil {
+		t.Errorf("Failed to close bolt: %s", err)
+	}
+
+	db, err := bolt.Open(path, 0600, nil)
+	if err != nil {
+		t.Errorf("Failed to open bolt: %s", err)
+	}
+	db.View(func(tx *bolt.Tx) error {
+		bkt := tx.Bucket([]byte(servername))
+		if bkt == nil {
+			t.Errorf("Meta bucket nof found")
+		}
+		return nil
+	})
+}
+
+func TestPutGetChangelog(t *testing.T) {
+	clog := "changelog-text"
+	log := logrus.NewEntry(&logrus.Logger{})
+	if err := SetupBolt(path, log); err != nil {
+		t.Errorf("Failed to setup bolt: %s", err)
+	}
+	defer os.Remove(path)
+
+	if err := DB.EnsureBuckets(meta); err != nil {
+		t.Errorf("Failed to ensure buckets: %s", err)
+	}
+	if err := DB.PutChangelog(servername, "apt", clog); err != nil {
+		t.Errorf("Failed to put changelog: %s", err)
+	}
+	if actual, err := DB.GetChangelog(servername, "apt"); err != nil {
+		t.Errorf("Failed to get changelog: %s", err)
+	} else {
+		if actual != clog {
+			t.Errorf("changelog is not same. e: %s, a: %s", clog, actual)
+		}
+	}
+}

cache/db.go

@@ -0,0 +1,56 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package cache
+
+import (
+	"github.com/future-architect/vuls/config"
+	"github.com/future-architect/vuls/models"
+)
+
+// DB has a cache instance
+var DB Cache
+
+const metabucket = "changelog-meta"
+
+// Cache is a interface of cache
+type Cache interface {
+	Close() error
+	GetMeta(string) (Meta, bool, error)
+	EnsureBuckets(Meta) error
+	PrettyPrint(Meta) error
+	GetChangelog(string, string) (string, error)
+	PutChangelog(string, string, string) error
+}
+
+// Meta holds a server name, distro information of the scanned server and
+// package information that was collected at the last scan.
+type Meta struct {
+	Name   string
+	Distro config.Distro
+	Packs  []models.PackageInfo
+}
+
+// FindPack search a PackageInfo
+func (m Meta) FindPack(name string) (pack models.PackageInfo, found bool) {
+	for _, p := range m.Packs {
+		if name == p.Name {
+			return p, true
+		}
+	}
+	return pack, false
+}

commands/configtest.go

@@ -0,0 +1,166 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package commands
+
+import (
+	"context"
+	"flag"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"github.com/Sirupsen/logrus"
+	"github.com/google/subcommands"
+
+	c "github.com/future-architect/vuls/config"
+	"github.com/future-architect/vuls/scan"
+	"github.com/future-architect/vuls/util"
+)
+
+// ConfigtestCmd is Subcommand
+type ConfigtestCmd struct {
+	configPath     string
+	askKeyPassword bool
+	sshExternal    bool
+
+	debug bool
+}
+
+// Name return subcommand name
+func (*ConfigtestCmd) Name() string { return "configtest" }
+
+// Synopsis return synopsis
+func (*ConfigtestCmd) Synopsis() string { return "Test configuration" }
+
+// Usage return usage
+func (*ConfigtestCmd) Usage() string {
+	return `configtest:
+	configtest
+		        [-config=/path/to/config.toml]
+	        	[-ask-key-password]
+	        	[-ssh-external]
+		        [-debug]
+
+		        [SERVER]...
+`
+}
+
+// SetFlags set flag
+func (p *ConfigtestCmd) SetFlags(f *flag.FlagSet) {
+	wd, _ := os.Getwd()
+	defaultConfPath := filepath.Join(wd, "config.toml")
+	f.StringVar(&p.configPath, "config", defaultConfPath, "/path/to/toml")
+
+	f.BoolVar(&p.debug, "debug", false, "debug mode")
+
+	f.BoolVar(
+		&p.askKeyPassword,
+		"ask-key-password",
+		false,
+		"Ask ssh privatekey password before scanning",
+	)
+
+	f.BoolVar(
+		&p.sshExternal,
+		"ssh-external",
+		false,
+		"Use external ssh command. Default: Use the Go native implementation")
+}
+
+// Execute execute
+func (p *ConfigtestCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
+
+	var keyPass string
+	var err error
+	if p.askKeyPassword {
+		prompt := "SSH key password: "
+		if keyPass, err = getPasswd(prompt); err != nil {
+			logrus.Error(err)
+			return subcommands.ExitFailure
+		}
+	}
+
+	c.Conf.Debug = p.debug
+	c.Conf.SSHExternal = p.sshExternal
+
+	err = c.Load(p.configPath, keyPass)
+	if err != nil {
+		logrus.Errorf("Error loading %s, %s", p.configPath, err)
+		return subcommands.ExitUsageError
+	}
+
+	var servernames []string
+	if 0 < len(f.Args()) {
+		servernames = f.Args()
+	} else {
+		stat, _ := os.Stdin.Stat()
+		if (stat.Mode() & os.ModeCharDevice) == 0 {
+			bytes, err := ioutil.ReadAll(os.Stdin)
+			if err != nil {
+				logrus.Errorf("Failed to read stdin: %s", err)
+				return subcommands.ExitFailure
+			}
+			fields := strings.Fields(string(bytes))
+			if 0 < len(fields) {
+				servernames = fields
+			}
+		}
+	}
+
+	target := make(map[string]c.ServerInfo)
+	for _, arg := range servernames {
+		found := false
+		for servername, info := range c.Conf.Servers {
+			if servername == arg {
+				target[servername] = info
+				found = true
+				break
+			}
+		}
+		if !found {
+			logrus.Errorf("%s is not in config", arg)
+			return subcommands.ExitUsageError
+		}
+	}
+	if 0 < len(servernames) {
+		c.Conf.Servers = target
+	}
+
+	// logger
+	Log := util.NewCustomLogger(c.ServerInfo{})
+
+	Log.Info("Validating Config...")
+	if !c.Conf.Validate() {
+		return subcommands.ExitUsageError
+	}
+
+	Log.Info("Detecting Server/Contianer OS... ")
+	if err := scan.InitServers(Log); err != nil {
+		Log.Errorf("Failed to init servers: %s", err)
+		return subcommands.ExitFailure
+	}
+
+	Log.Info("Checking sudo configuration... ")
+	if err := scan.CheckIfSudoNoPasswd(Log); err != nil {
+		Log.Errorf("Failed to sudo with nopassword via SSH. Define NOPASSWD in /etc/sudoers on target servers. err: %s", err)
+		return subcommands.ExitFailure
+	}
+	scan.PrintSSHableServerNames()
+	return subcommands.ExitSuccess
+}

commands/discover.go

@@ -18,6 +18,7 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 package commands
 
 import (
+	"context"
 	"flag"
 	"fmt"
 	"os"
@@ -25,7 +26,6 @@ import (
 	"text/template"
 
 	"github.com/google/subcommands"
-	"golang.org/x/net/context"
 
 	"github.com/Sirupsen/logrus"
 	ps "github.com/kotakanbe/go-pingscanner"
@@ -93,14 +93,14 @@ func printConfigToml(ips []string) (err error) {
 [slack]
 hookURL      = "https://hooks.slack.com/services/abc123/defghijklmnopqrstuvwxyz"
 channel      = "#channel-name"
-#channel      = "#{servername}"
+#channel      = "${servername}"
 iconEmoji    = ":ghost:"
 authUser     = "username"
 notifyUsers  = ["@username"]
 
 [mail]
 smtpAddr      = "smtp.gmail.com"
-smtpPort      = 465
+smtpPort      = "587"
 user          = "username"
 password      = "password"
 from          = "from@address.com"
@@ -112,6 +112,15 @@ subjectPrefix = "[vuls]"
 #port        = "22"
 #user        = "username"
 #keyPath     = "/home/username/.ssh/id_rsa"
+#cpeNames = [
+#  "cpe:/a:rubyonrails:ruby_on_rails:4.2.1",
+#]
+#dependencyCheckXMLPath = "/tmp/dependency-check-report.xml"
+#containers = ["${running}"]
+#ignoreCves = ["CVE-2014-6271"]
+#optional = [
+#    ["key", "value"],
+#]
 
 [servers]
 {{- $names:=  .Names}}
@@ -124,6 +133,12 @@ host         = "{{$ip}}"
 #cpeNames = [
 #  "cpe:/a:rubyonrails:ruby_on_rails:4.2.1",
 #]
+#dependencyCheckXMLPath = "/tmp/dependency-check-report.xml"
+#containers = ["${running}"]
+#ignoreCves = ["CVE-2014-0160"]
+#optional = [
+#    ["key", "value"],
+#]
 {{end}}
 
 `

commands/history.go

@@ -0,0 +1,99 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package commands
+
+import (
+	"context"
+	"flag"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strings"
+
+	c "github.com/future-architect/vuls/config"
+	"github.com/future-architect/vuls/report"
+	"github.com/google/subcommands"
+)
+
+// HistoryCmd is Subcommand of list scanned results
+type HistoryCmd struct {
+	debug      bool
+	debugSQL   bool
+	resultsDir string
+}
+
+// Name return subcommand name
+func (*HistoryCmd) Name() string { return "history" }
+
+// Synopsis return synopsis
+func (*HistoryCmd) Synopsis() string {
+	return `List history of scanning.`
+}
+
+// Usage return usage
+func (*HistoryCmd) Usage() string {
+	return `history:
+	history
+		[-results-dir=/path/to/results]
+	`
+}
+
+// SetFlags set flag
+func (p *HistoryCmd) SetFlags(f *flag.FlagSet) {
+	f.BoolVar(&p.debugSQL, "debug-sql", false, "SQL debug mode")
+
+	wd, _ := os.Getwd()
+	defaultResultsDir := filepath.Join(wd, "results")
+	f.StringVar(&p.resultsDir, "results-dir", defaultResultsDir, "/path/to/results")
+}
+
+// Execute execute
+func (p *HistoryCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
+
+	c.Conf.DebugSQL = p.debugSQL
+	c.Conf.ResultsDir = p.resultsDir
+
+	var err error
+	var jsonDirs report.JSONDirs
+	if jsonDirs, err = report.GetValidJSONDirs(); err != nil {
+		return subcommands.ExitFailure
+	}
+	for _, d := range jsonDirs {
+		var files []os.FileInfo
+		if files, err = ioutil.ReadDir(d); err != nil {
+			return subcommands.ExitFailure
+		}
+		var hosts []string
+		for _, f := range files {
+			if filepath.Ext(f.Name()) != ".json" {
+				continue
+			}
+			fileBase := strings.TrimSuffix(f.Name(), filepath.Ext(f.Name()))
+			hosts = append(hosts, fileBase)
+		}
+		splitPath := strings.Split(d, string(os.PathSeparator))
+		timeStr := splitPath[len(splitPath)-1]
+		fmt.Printf("%s scanned %d servers: %s\n",
+			timeStr,
+			len(hosts),
+			strings.Join(hosts, ", "),
+		)
+	}
+	return subcommands.ExitSuccess
+}

commands/prepare.go

@@ -18,15 +18,16 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 package commands
 
 import (
+	"context"
 	"flag"
 	"os"
+	"path/filepath"
 
 	"github.com/Sirupsen/logrus"
 	c "github.com/future-architect/vuls/config"
 	"github.com/future-architect/vuls/scan"
 	"github.com/future-architect/vuls/util"
 	"github.com/google/subcommands"
-	"golang.org/x/net/context"
 )
 
 // PrepareCmd is Subcommand of host discovery mode
@@ -37,7 +38,7 @@ type PrepareCmd struct {
 	askSudoPassword bool
 	askKeyPassword  bool
 
-	useUnattendedUpgrades bool
+	sshExternal bool
 }
 
 // Name return subcommand name
@@ -45,7 +46,6 @@ func (*PrepareCmd) Name() string { return "prepare" }
 
 // Synopsis return synopsis
 func (*PrepareCmd) Synopsis() string {
-	//  return "Install packages Ubuntu: unattended-upgrade, CentOS: yum-plugin-security)"
 	return `Install required packages to scan.
 				CentOS: yum-plugin-security, yum-plugin-changelog
 				Amazon: None
@@ -60,10 +60,11 @@ func (*PrepareCmd) Usage() string {
 	return `prepare:
 	prepare
 			[-config=/path/to/config.toml]
-			[-ask-sudo-password]
 			[-ask-key-password]
 			[-debug]
+			[-ssh-external]
 
+			[SERVER]...
 `
 }
 
@@ -72,7 +73,9 @@ func (p *PrepareCmd) SetFlags(f *flag.FlagSet) {
 
 	f.BoolVar(&p.debug, "debug", false, "debug mode")
 
-	defaultConfPath := os.Getenv("PWD") + "/config.toml"
+	wd, _ := os.Getwd()
+
+	defaultConfPath := filepath.Join(wd, "config.toml")
 	f.StringVar(&p.configPath, "config", defaultConfPath, "/path/to/toml")
 
 	f.BoolVar(
@@ -86,20 +89,20 @@ func (p *PrepareCmd) SetFlags(f *flag.FlagSet) {
 		&p.askSudoPassword,
 		"ask-sudo-password",
 		false,
-		"Ask sudo password of target servers before scanning",
+		"[Deprecated] THIS OPTION WAS REMOVED FOR SECURITY REASONS. Define NOPASSWD in /etc/sudoers on target servers and use SSH key-based authentication",
 	)
 
 	f.BoolVar(
-		&p.useUnattendedUpgrades,
-		"use-unattended-upgrades",
+		&p.sshExternal,
+		"ssh-external",
 		false,
-		"[Depricated] For Ubuntu, install unattended-upgrades",
-	)
+		"Use external ssh command. Default: Use the Go native implementation")
+
 }
 
 // Execute execute
 func (p *PrepareCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
-	var keyPass, sudoPass string
+	var keyPass string
 	var err error
 	if p.askKeyPassword {
 		prompt := "SSH key password: "
@@ -109,14 +112,11 @@ func (p *PrepareCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{
 		}
 	}
 	if p.askSudoPassword {
-		prompt := "sudo password: "
-		if sudoPass, err = getPasswd(prompt); err != nil {
-			logrus.Error(err)
-			return subcommands.ExitFailure
-		}
+		logrus.Errorf("[Deprecated] -ask-sudo-password WAS REMOVED FOR SECURITY REASONS. Define NOPASSWD in /etc/sudoers on target servers and use SSH key-based authentication")
+		return subcommands.ExitFailure
 	}
 
-	err = c.Load(p.configPath, keyPass, sudoPass)
+	err = c.Load(p.configPath, keyPass)
 	if err != nil {
 		logrus.Errorf("Error loading %s, %s", p.configPath, err)
 		return subcommands.ExitUsageError
@@ -143,26 +143,29 @@ func (p *PrepareCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{
 	}
 
 	c.Conf.Debug = p.debug
-	c.Conf.UseUnattendedUpgrades = p.useUnattendedUpgrades
+	c.Conf.SSHExternal = p.sshExternal
 
 	// Set up custom logger
 	logger := util.NewCustomLogger(c.ServerInfo{})
 
 	logger.Info("Detecting OS... ")
-	err = scan.InitServers(logger)
-	if err != nil {
-		logger.Errorf("Failed to init servers. err: %s", err)
+	if err := scan.InitServers(logger); err != nil {
+		logger.Errorf("Failed to init servers: %s", err)
+		return subcommands.ExitFailure
+	}
+
+	logger.Info("Checking sudo configuration... ")
+	if err := scan.CheckIfSudoNoPasswd(logger); err != nil {
+		logger.Errorf("Failed to sudo with nopassword via SSH. Define NOPASSWD in /etc/sudoers on target servers")
 		return subcommands.ExitFailure
 	}
 
-	logger.Info("Installing...")
 	if errs := scan.Prepare(); 0 < len(errs) {
 		for _, e := range errs {
-			logger.Errorf("Failed: %s", e)
+			logger.Errorf("Failed to prepare: %s", e)
 		}
 		return subcommands.ExitFailure
 	}
 
-	logger.Info("Success")
 	return subcommands.ExitSuccess
 }

commands/scan.go

@@ -18,18 +18,23 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 package commands
 
 import (
+	"context"
 	"flag"
+	"fmt"
+	"io/ioutil"
 	"os"
+	"path/filepath"
+	"strings"
+	"time"
 
 	"github.com/Sirupsen/logrus"
 	c "github.com/future-architect/vuls/config"
 	"github.com/future-architect/vuls/cveapi"
-	"github.com/future-architect/vuls/db"
 	"github.com/future-architect/vuls/report"
 	"github.com/future-architect/vuls/scan"
 	"github.com/future-architect/vuls/util"
 	"github.com/google/subcommands"
-	"golang.org/x/net/context"
+	"github.com/k0kubun/pp"
 )
 
 // ScanCmd is Subcommand of host discovery mode
@@ -40,20 +45,40 @@ type ScanCmd struct {
 
 	configPath string
 
-	dbpath           string
+	resultsDir       string
+	cvedbtype        string
+	cvedbpath        string
 	cveDictionaryURL string
-	cvssScoreOver    float64
-	httpProxy        string
+	cacheDBPath      string
 
-	// reporting
-	reportSlack bool
-	reportMail  bool
+	cvssScoreOver      float64
+	ignoreUnscoredCves bool
 
+	httpProxy       string
 	askSudoPassword bool
 	askKeyPassword  bool
 
-	useYumPluginSecurity  bool
-	useUnattendedUpgrades bool
+	containersOnly bool
+	skipBroken     bool
+
+	// reporting
+	reportSlack     bool
+	reportMail      bool
+	reportJSON      bool
+	reportText      bool
+	reportS3        bool
+	reportAzureBlob bool
+	reportXML       bool
+
+	awsProfile  string
+	awsS3Bucket string
+	awsRegion   string
+
+	azureAccount   string
+	azureKey       string
+	azureContainer string
+
+	sshExternal bool
 }
 
 // Name return subcommand name
@@ -68,16 +93,35 @@ func (*ScanCmd) Usage() string {
 	scan
 		[-lang=en|ja]
 		[-config=/path/to/config.toml]
-		[-dbpath=/path/to/vuls.sqlite3]
+		[-results-dir=/path/to/results]
+		[-cve-dictionary-dbtype=sqlite3|mysql]
+		[-cve-dictionary-dbpath=/path/to/cve.sqlite3 or mysql connection string]
 		[-cve-dictionary-url=http://127.0.0.1:1323]
+		[-cache-dbpath=/path/to/cache.db]
 		[-cvss-over=7]
-		[-report-slack]
+		[-ignore-unscored-cves]
+		[-ssh-external]
+		[-containers-only]
+		[-skip-broken]
+		[-report-azure-blob]
+		[-report-json]
 		[-report-mail]
+		[-report-s3]
+		[-report-slack]
+		[-report-text]
+		[-report-xml]
 		[-http-proxy=http://192.168.0.1:8080]
-		[-ask-sudo-password]
 		[-ask-key-password]
 		[-debug]
 		[-debug-sql]
+		[-aws-profile=default]
+		[-aws-region=us-west-2]
+		[-aws-s3-bucket=bucket_name]
+		[-azure-account=accout]
+		[-azure-key=key]
+		[-azure-container=container]
+
+		[SERVER]...
 `
 }
 
@@ -87,11 +131,25 @@ func (p *ScanCmd) SetFlags(f *flag.FlagSet) {
 	f.BoolVar(&p.debug, "debug", false, "debug mode")
 	f.BoolVar(&p.debugSQL, "debug-sql", false, "SQL debug mode")
 
-	defaultConfPath := os.Getenv("PWD") + "/config.toml"
+	wd, _ := os.Getwd()
+
+	defaultConfPath := filepath.Join(wd, "config.toml")
 	f.StringVar(&p.configPath, "config", defaultConfPath, "/path/to/toml")
 
-	defaultDBPath := os.Getenv("PWD") + "/vuls.sqlite3"
-	f.StringVar(&p.dbpath, "dbpath", defaultDBPath, "/path/to/sqlite3")
+	defaultResultsDir := filepath.Join(wd, "results")
+	f.StringVar(&p.resultsDir, "results-dir", defaultResultsDir, "/path/to/results")
+
+	f.StringVar(
+		&p.cvedbtype,
+		"cve-dictionary-dbtype",
+		"sqlite3",
+		"DB type for fetching CVE dictionary (sqlite3 or mysql)")
+
+	f.StringVar(
+		&p.cvedbpath,
+		"cve-dictionary-dbpath",
+		"",
+		"/path/to/sqlite3 (For get cve detail from cve.sqlite3)")
 
 	defaultURL := "http://127.0.0.1:1323"
 	f.StringVar(
@@ -100,12 +158,43 @@ func (p *ScanCmd) SetFlags(f *flag.FlagSet) {
 		defaultURL,
 		"http://CVE.Dictionary")
 
+	defaultCacheDBPath := filepath.Join(wd, "cache.db")
+	f.StringVar(
+		&p.cacheDBPath,
+		"cache-dbpath",
+		defaultCacheDBPath,
+		"/path/to/cache.db (local cache of changelog for Ubuntu/Debian)")
+
 	f.Float64Var(
 		&p.cvssScoreOver,
 		"cvss-over",
 		0,
 		"-cvss-over=6.5 means reporting CVSS Score 6.5 and over (default: 0 (means report all))")
 
+	f.BoolVar(
+		&p.ignoreUnscoredCves,
+		"ignore-unscored-cves",
+		false,
+		"Don't report the unscored CVEs")
+
+	f.BoolVar(
+		&p.sshExternal,
+		"ssh-external",
+		false,
+		"Use external ssh command. Default: Use the Go native implementation")
+
+	f.BoolVar(
+		&p.containersOnly,
+		"containers-only",
+		false,
+		"Scan containers only. Default: Scan both of hosts and containers")
+
+	f.BoolVar(
+		&p.skipBroken,
+		"skip-broken",
+		false,
+		"[For CentOS] yum update changelog with --skip-broken option")
+
 	f.StringVar(
 		&p.httpProxy,
 		"http-proxy",
@@ -113,8 +202,41 @@ func (p *ScanCmd) SetFlags(f *flag.FlagSet) {
 		"http://proxy-url:port (default: empty)",
 	)
 
-	f.BoolVar(&p.reportSlack, "report-slack", false, "Slack report")
-	f.BoolVar(&p.reportMail, "report-mail", false, "Email report")
+	f.BoolVar(&p.reportSlack, "report-slack", false, "Send report via Slack")
+	f.BoolVar(&p.reportMail, "report-mail", false, "Send report via Email")
+	f.BoolVar(&p.reportJSON,
+		"report-json",
+		false,
+		fmt.Sprintf("Write report to JSON files (%s/results/current)", wd),
+	)
+	f.BoolVar(&p.reportText,
+		"report-text",
+		false,
+		fmt.Sprintf("Write report to text files (%s/results/current)", wd),
+	)
+	f.BoolVar(&p.reportXML,
+		"report-xml",
+		false,
+		fmt.Sprintf("Write report to XML files (%s/results/current)", wd),
+	)
+
+	f.BoolVar(&p.reportS3,
+		"report-s3",
+		false,
+		"Write report to S3 (bucket/yyyyMMdd_HHmm/servername.json)",
+	)
+	f.StringVar(&p.awsProfile, "aws-profile", "default", "AWS profile to use")
+	f.StringVar(&p.awsRegion, "aws-region", "us-east-1", "AWS region to use")
+	f.StringVar(&p.awsS3Bucket, "aws-s3-bucket", "", "S3 bucket name")
+
+	f.BoolVar(&p.reportAzureBlob,
+		"report-azure-blob",
+		false,
+		"Write report to Azure Storage blob (container/yyyyMMdd_HHmm/servername.json)",
+	)
+	f.StringVar(&p.azureAccount, "azure-account", "", "Azure account name to use. AZURE_STORAGE_ACCOUNT environment variable is used if not specified")
+	f.StringVar(&p.azureKey, "azure-key", "", "Azure account key to use. AZURE_STORAGE_ACCESS_KEY environment variable is used if not specified")
+	f.StringVar(&p.azureContainer, "azure-container", "", "Azure storage container name")
 
 	f.BoolVar(
 		&p.askKeyPassword,
@@ -127,28 +249,13 @@ func (p *ScanCmd) SetFlags(f *flag.FlagSet) {
 		&p.askSudoPassword,
 		"ask-sudo-password",
 		false,
-		"Ask sudo password of target servers before scanning",
+		"[Deprecated] THIS OPTION WAS REMOVED FOR SECURITY REASONS. Define NOPASSWD in /etc/sudoers on target servers and use SSH key-based authentication",
 	)
-
-	f.BoolVar(
-		&p.useYumPluginSecurity,
-		"use-yum-plugin-security",
-		false,
-		"[Depricated] For CentOS 5. Scan by yum-plugin-security or not (use yum check-update by default)",
-	)
-
-	f.BoolVar(
-		&p.useUnattendedUpgrades,
-		"use-unattended-upgrades",
-		false,
-		"[Depricated] For Ubuntu. Scan by unattended-upgrades or not (use apt-get upgrade --dry-run by default)",
-	)
-
 }
 
 // Execute execute
 func (p *ScanCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
-	var keyPass, sudoPass string
+	var keyPass string
 	var err error
 	if p.askKeyPassword {
 		prompt := "SSH key password: "
@@ -158,22 +265,47 @@ func (p *ScanCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{})
 		}
 	}
 	if p.askSudoPassword {
-		prompt := "sudo password: "
-		if sudoPass, err = getPasswd(prompt); err != nil {
-			logrus.Error(err)
-			return subcommands.ExitFailure
-		}
+		logrus.Errorf("[Deprecated] -ask-sudo-password WAS REMOVED FOR SECURITY REASONS. Define NOPASSWD in /etc/sudoers on target servers and use SSH key-based authentication")
+		return subcommands.ExitFailure
 	}
 
-	err = c.Load(p.configPath, keyPass, sudoPass)
+	c.Conf.Debug = p.debug
+	err = c.Load(p.configPath, keyPass)
 	if err != nil {
 		logrus.Errorf("Error loading %s, %s", p.configPath, err)
 		return subcommands.ExitUsageError
 	}
 
-	logrus.Infof("Start scanning (config: %s)", p.configPath)
+	logrus.Info("Start scanning")
+	logrus.Infof("config: %s", p.configPath)
+	if p.cvedbpath != "" {
+		if p.cvedbtype == "sqlite3" {
+			logrus.Infof("cve-dictionary: %s", p.cvedbpath)
+		}
+	} else {
+		logrus.Infof("cve-dictionary: %s", p.cveDictionaryURL)
+	}
+
+	var servernames []string
+	if 0 < len(f.Args()) {
+		servernames = f.Args()
+	} else {
+		stat, _ := os.Stdin.Stat()
+		if (stat.Mode() & os.ModeCharDevice) == 0 {
+			bytes, err := ioutil.ReadAll(os.Stdin)
+			if err != nil {
+				logrus.Errorf("Failed to read stdin: %s", err)
+				return subcommands.ExitFailure
+			}
+			fields := strings.Fields(string(bytes))
+			if 0 < len(fields) {
+				servernames = fields
+			}
+		}
+	}
+
 	target := make(map[string]c.ServerInfo)
-	for _, arg := range f.Args() {
+	for _, arg := range servernames {
 		found := false
 		for servername, info := range c.Conf.Servers {
 			if servername == arg {
@@ -187,20 +319,21 @@ func (p *ScanCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{})
 			return subcommands.ExitUsageError
 		}
 	}
-	if 0 < len(f.Args()) {
+	if 0 < len(servernames) {
 		c.Conf.Servers = target
 	}
+	logrus.Debugf("%s", pp.Sprintf("%v", target))
 
 	c.Conf.Lang = p.lang
-	c.Conf.Debug = p.debug
 	c.Conf.DebugSQL = p.debugSQL
 
 	// logger
 	Log := util.NewCustomLogger(c.ServerInfo{})
+	scannedAt := time.Now()
 
 	// report
 	reports := []report.ResultWriter{
-		report.TextWriter{},
+		report.StdoutWriter{},
 		report.LogrusWriter{},
 	}
 	if p.reportSlack {
@@ -209,12 +342,61 @@ func (p *ScanCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{})
 	if p.reportMail {
 		reports = append(reports, report.MailWriter{})
 	}
+	if p.reportJSON {
+		reports = append(reports, report.JSONWriter{ScannedAt: scannedAt})
+	}
+	if p.reportText {
+		reports = append(reports, report.TextFileWriter{ScannedAt: scannedAt})
+	}
+	if p.reportXML {
+		reports = append(reports, report.XMLWriter{ScannedAt: scannedAt})
+	}
+	if p.reportS3 {
+		c.Conf.AwsRegion = p.awsRegion
+		c.Conf.AwsProfile = p.awsProfile
+		c.Conf.S3Bucket = p.awsS3Bucket
+		if err := report.CheckIfBucketExists(); err != nil {
+			Log.Errorf("Failed to access to the S3 bucket. err: %s", err)
+			Log.Error("Ensure the bucket or check AWS config before scanning")
+			return subcommands.ExitUsageError
+		}
+		reports = append(reports, report.S3Writer{})
+	}
+	if p.reportAzureBlob {
+		c.Conf.AzureAccount = p.azureAccount
+		if len(c.Conf.AzureAccount) == 0 {
+			c.Conf.AzureAccount = os.Getenv("AZURE_STORAGE_ACCOUNT")
+		}
 
-	c.Conf.DBPath = p.dbpath
+		c.Conf.AzureKey = p.azureKey
+		if len(c.Conf.AzureKey) == 0 {
+			c.Conf.AzureKey = os.Getenv("AZURE_STORAGE_ACCESS_KEY")
+		}
+
+		c.Conf.AzureContainer = p.azureContainer
+		if len(c.Conf.AzureContainer) == 0 {
+			Log.Error("Azure storage container name is requied with --azure-container option")
+			return subcommands.ExitUsageError
+		}
+		if err := report.CheckIfAzureContainerExists(); err != nil {
+			Log.Errorf("Failed to access to the Azure Blob container. err: %s", err)
+			Log.Error("Ensure the container or check Azure config before scanning")
+			return subcommands.ExitUsageError
+		}
+		reports = append(reports, report.AzureBlobWriter{})
+	}
+
+	c.Conf.ResultsDir = p.resultsDir
+	c.Conf.CveDBType = p.cvedbtype
+	c.Conf.CveDBPath = p.cvedbpath
 	c.Conf.CveDictionaryURL = p.cveDictionaryURL
+	c.Conf.CacheDBPath = p.cacheDBPath
+	c.Conf.CvssScoreOver = p.cvssScoreOver
+	c.Conf.IgnoreUnscoredCves = p.ignoreUnscoredCves
+	c.Conf.SSHExternal = p.sshExternal
 	c.Conf.HTTPProxy = p.httpProxy
-	c.Conf.UseYumPluginSecurity = p.useYumPluginSecurity
-	c.Conf.UseUnattendedUpgrades = p.useUnattendedUpgrades
+	c.Conf.ContainersOnly = p.containersOnly
+	c.Conf.SkipBroken = p.skipBroken
 
 	Log.Info("Validating Config...")
 	if !c.Conf.Validate() {
@@ -222,18 +404,26 @@ func (p *ScanCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{})
 	}
 
 	if ok, err := cveapi.CveClient.CheckHealth(); !ok {
-		Log.Errorf("CVE HTTP server is not running. %#v", cveapi.CveClient)
-		Log.Fatal(err)
+		Log.Errorf("CVE HTTP server is not running. err: %s", err)
+		Log.Errorf("Run go-cve-dictionary as server mode or specify -cve-dictionary-dbpath option")
 		return subcommands.ExitFailure
 	}
 
-	Log.Info("Detecting the type of OS... ")
-	err = scan.InitServers(Log)
-	if err != nil {
-		Log.Errorf("Failed to init servers. Check the configuration. err: %s", err)
+	Log.Info("Detecting Server/Contianer OS... ")
+	if err := scan.InitServers(Log); err != nil {
+		Log.Errorf("Failed to init servers: %s", err)
 		return subcommands.ExitFailure
 	}
 
+	Log.Info("Checking sudo configuration... ")
+	if err := scan.CheckIfSudoNoPasswd(Log); err != nil {
+		Log.Errorf("Failed to sudo with nopassword via SSH. Define NOPASSWD in /etc/sudoers on target servers")
+		return subcommands.ExitFailure
+	}
+
+	Log.Info("Detecting Platforms... ")
+	scan.DetectPlatforms(Log)
+
 	Log.Info("Scanning vulnerabilities... ")
 	if errs := scan.Scan(); 0 < len(errs) {
 		for _, e := range errs {
@@ -252,25 +442,10 @@ func (p *ScanCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{})
 	filtered := scanResults.FilterByCvssOver()
 	for _, w := range reports {
 		if err := w.Write(filtered); err != nil {
-			Log.Fatalf("Failed to output report, err: %s", err)
+			Log.Fatalf("Failed to report, err: %s", err)
 			return subcommands.ExitFailure
 		}
 	}
 
-	Log.Info("Insert to DB...")
-	if err := db.OpenDB(); err != nil {
-		Log.Errorf("Failed to open DB. datafile: %s, err: %s", c.Conf.DBPath, err)
-		return subcommands.ExitFailure
-	}
-	if err := db.MigrateDB(); err != nil {
-		Log.Errorf("Failed to migrate. err: %s", err)
-		return subcommands.ExitFailure
-	}
-
-	if err := db.Insert(scanResults); err != nil {
-		Log.Fatalf("Failed to insert. dbpath: %s, err: %s", c.Conf.DBPath, err)
-		return subcommands.ExitFailure
-	}
-
 	return subcommands.ExitSuccess
 }

commands/tui.go

@@ -18,21 +18,24 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 package commands
 
 import (
+	"context"
 	"flag"
-	"fmt"
+	"io/ioutil"
 	"os"
+	"path/filepath"
+	"strings"
 
+	log "github.com/Sirupsen/logrus"
 	c "github.com/future-architect/vuls/config"
 	"github.com/future-architect/vuls/report"
 	"github.com/google/subcommands"
-	"golang.org/x/net/context"
 )
 
 // TuiCmd is Subcommand of host discovery mode
 type TuiCmd struct {
-	lang     string
-	debugSQL bool
-	dbpath   string
+	lang       string
+	debugSQL   bool
+	resultsDir string
 }
 
 // Name return subcommand name
@@ -44,7 +47,7 @@ func (*TuiCmd) Synopsis() string { return "Run Tui view to anayze vulnerabilites
 // Usage return usage
 func (*TuiCmd) Usage() string {
 	return `tui:
-	tui [-dbpath=/path/to/vuls.sqlite3]
+	tui [-results-dir=/path/to/results]
 
 `
 }
@@ -54,15 +57,49 @@ func (p *TuiCmd) SetFlags(f *flag.FlagSet) {
 	//  f.StringVar(&p.lang, "lang", "en", "[en|ja]")
 	f.BoolVar(&p.debugSQL, "debug-sql", false, "debug SQL")
 
-	defaultDBPath := os.Getenv("PWD") + "/vuls.sqlite3"
-	f.StringVar(&p.dbpath, "dbpath", defaultDBPath,
-		fmt.Sprintf("/path/to/sqlite3 (default: %s)", defaultDBPath))
+	wd, _ := os.Getwd()
+
+	defaultResultsDir := filepath.Join(wd, "results")
+	f.StringVar(&p.resultsDir, "results-dir", defaultResultsDir, "/path/to/results")
 }
 
 // Execute execute
 func (p *TuiCmd) Execute(_ context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
 	c.Conf.Lang = "en"
 	c.Conf.DebugSQL = p.debugSQL
-	c.Conf.DBPath = p.dbpath
-	return report.RunTui()
+	c.Conf.ResultsDir = p.resultsDir
+
+	var jsonDirName string
+	var err error
+	if 0 < len(f.Args()) {
+		var jsonDirs report.JSONDirs
+		if jsonDirs, err = report.GetValidJSONDirs(); err != nil {
+			return subcommands.ExitFailure
+		}
+		for _, d := range jsonDirs {
+			splitPath := strings.Split(d, string(os.PathSeparator))
+			if splitPath[len(splitPath)-1] == f.Args()[0] {
+				jsonDirName = f.Args()[0]
+				break
+			}
+		}
+		if len(jsonDirName) == 0 {
+			log.Errorf("First Argument have to be JSON directory name : %s", err)
+			return subcommands.ExitFailure
+		}
+	} else {
+		stat, _ := os.Stdin.Stat()
+		if (stat.Mode() & os.ModeCharDevice) == 0 {
+			bytes, err := ioutil.ReadAll(os.Stdin)
+			if err != nil {
+				log.Errorf("Failed to read stdin: %s", err)
+				return subcommands.ExitFailure
+			}
+			fields := strings.Fields(string(bytes))
+			if 0 < len(fields) {
+				jsonDirName = fields[0]
+			}
+		}
+	}
+	return report.RunTui(jsonDirName)
 }

config/config.go

@@ -41,23 +41,65 @@ type Config struct {
 
 	CveDictionaryURL string `valid:"url"`
 
-	CvssScoreOver float64
-	HTTPProxy     string `valid:"url"`
-	DBPath        string
+	CvssScoreOver      float64
+	IgnoreUnscoredCves bool
+
+	SSHExternal    bool
+	ContainersOnly bool
+	SkipBroken     bool
+
+	HTTPProxy   string `valid:"url"`
+	ResultsDir  string
+	CveDBType   string
+	CveDBPath   string
+	CacheDBPath string
+
+	AwsProfile string
+	AwsRegion  string
+	S3Bucket   string
+
+	AzureAccount   string
+	AzureKey       string
+	AzureContainer string
+
 	//  CpeNames      []string
 	//  SummaryMode          bool
-	UseYumPluginSecurity  bool
-	UseUnattendedUpgrades bool
 }
 
 // Validate configuration
 func (c Config) Validate() bool {
 	errs := []error{}
 
-	if len(c.DBPath) != 0 {
-		if ok, _ := valid.IsFilePath(c.DBPath); !ok {
+	if len(c.ResultsDir) != 0 {
+		if ok, _ := valid.IsFilePath(c.ResultsDir); !ok {
 			errs = append(errs, fmt.Errorf(
-				"SQLite3 DB path must be a *Absolute* file path. dbpath: %s", c.DBPath))
+				"JSON base directory must be a *Absolute* file path. -results-dir: %s", c.ResultsDir))
+		}
+	}
+
+	// If no valid DB type is set, default to sqlite3
+	if c.CveDBType == "" {
+		c.CveDBType = "sqlite3"
+	}
+
+	if c.CveDBType != "sqlite3" && c.CveDBType != "mysql" {
+		errs = append(errs, fmt.Errorf(
+			"CVE DB type must be either 'sqlite3' or 'mysql'.  -cve-dictionary-dbtype: %s", c.CveDBType))
+	}
+
+	if c.CveDBType == "sqlite3" {
+		if len(c.CveDBPath) != 0 {
+			if ok, _ := valid.IsFilePath(c.CveDBPath); !ok {
+				errs = append(errs, fmt.Errorf(
+					"SQLite3 DB(Cve Dictionary) path must be a *Absolute* file path. -cve-dictionary-dbpath: %s", c.CveDBPath))
+			}
+		}
+	}
+
+	if len(c.CacheDBPath) != 0 {
+		if ok, _ := valid.IsFilePath(c.CacheDBPath); !ok {
+			errs = append(errs, fmt.Errorf(
+				"Cache DB path must be a *Absolute* file path. -cache-dbpath: %s", c.CacheDBPath))
 		}
 	}
 
@@ -160,7 +202,6 @@ type SlackConf struct {
 
 // Validate validates configuration
 func (c *SlackConf) Validate() (errs []error) {
-
 	if !c.UseThisTime {
 		return
 	}
@@ -188,8 +229,6 @@ func (c *SlackConf) Validate() (errs []error) {
 		errs = append(errs, err)
 	}
 
-	// TODO check if slack configration is valid
-
 	return
 }
 
@@ -197,25 +236,63 @@ func (c *SlackConf) Validate() (errs []error) {
 type ServerInfo struct {
 	ServerName  string
 	User        string
-	Password    string
 	Host        string
 	Port        string
 	KeyPath     string
 	KeyPassword string
-	SudoOpt     SudoOption
 
-	CpeNames []string
+	CpeNames               []string
+	DependencyCheckXMLPath string
 
-	// DebugLog Color
-	LogMsgAnsiColor string
+	// Container Names or IDs
+	Containers []string
+
+	IgnoreCves []string
+
+	// Optional key-value set that will be outputted to JSON
+	Optional [][]interface{}
+
+	// For CentOS, RHEL, Amazon
+	Enablerepo string
+
+	// used internal
+	LogMsgAnsiColor string // DebugLog Color
+	Container       Container
+	Distro          Distro
 }
 
-// SudoOption is flag of sudo option.
-type SudoOption struct {
-
-	// echo pass | sudo -S ls
-	ExecBySudo bool
-
-	// echo pass | sudo sh -C 'ls'
-	ExecBySudoSh bool
+// GetServerName returns ServerName if this serverInfo is about host.
+// If this serverInfo is abount a container, returns containerID@ServerName
+func (s ServerInfo) GetServerName() string {
+	if len(s.Container.ContainerID) == 0 {
+		return s.ServerName
+	}
+	return fmt.Sprintf("%s@%s", s.Container.ContainerID, s.ServerName)
+}
+
+// Distro has distribution info
+type Distro struct {
+	Family  string
+	Release string
+}
+
+func (l Distro) String() string {
+	return fmt.Sprintf("%s %s", l.Family, l.Release)
+}
+
+// IsContainer returns whether this ServerInfo is about container
+func (s ServerInfo) IsContainer() bool {
+	return 0 < len(s.Container.ContainerID)
+}
+
+// SetContainer set container
+func (s *ServerInfo) SetContainer(d Container) {
+	s.Container = d
+}
+
+// Container has Container information.
+type Container struct {
+	ContainerID string
+	Name        string
+	Type        string
 }

config/loader.go

@@ -18,14 +18,13 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 package config
 
 // Load loads configuration
-func Load(path, keyPass, sudoPass string) error {
+func Load(path, keyPass string) error {
 	var loader Loader
 	loader = TOMLLoader{}
-
-	return loader.Load(path, keyPass, sudoPass)
+	return loader.Load(path, keyPass)
 }
 
 // Loader is interface of concrete loader
 type Loader interface {
-	Load(string, string, string) error
+	Load(string, string) error
 }

config/tomlloader.go

@@ -20,10 +20,11 @@ package config
 import (
 	"fmt"
 	"os"
+	"strings"
 
 	"github.com/BurntSushi/toml"
 	log "github.com/Sirupsen/logrus"
-	"github.com/k0kubun/pp"
+	"github.com/future-architect/vuls/contrib/owasp-dependency-check/parser"
 )
 
 // TOMLLoader loads config
@@ -31,7 +32,11 @@ type TOMLLoader struct {
 }
 
 // Load load the configuraiton TOML file specified by path arg.
-func (c TOMLLoader) Load(pathToToml, keyPass, sudoPass string) (err error) {
+func (c TOMLLoader) Load(pathToToml, keyPass string) error {
+	if Conf.Debug {
+		log.SetLevel(log.DebugLevel)
+	}
+
 	var conf Config
 	if _, err := toml.DecodeFile(pathToToml, &conf); err != nil {
 		log.Error("Load config failed", err)
@@ -49,50 +54,51 @@ func (c TOMLLoader) Load(pathToToml, keyPass, sudoPass string) (err error) {
 		d.KeyPassword = keyPass
 	}
 
-	if sudoPass != "" {
-		d.Password = sudoPass
-	}
-
 	i := 0
 	for name, v := range conf.Servers {
-
-		if 0 < len(v.KeyPassword) || 0 < len(v.Password) {
-			log.Warn("[Depricated] password and keypassword in config file are unsecure. Remove them immediately for a security reason. They will be removed in a future release.")
+		if 0 < len(v.KeyPassword) {
+			log.Warn("[Deprecated] KEYPASSWORD IN CONFIG FILE ARE UNSECURE. REMOVE THEM IMMEDIATELY FOR A SECURITY REASONS. THEY WILL BE REMOVED IN A FUTURE RELEASE.")
 		}
 
 		s := ServerInfo{ServerName: name}
-		s.User = v.User
-		if s.User == "" {
-			s.User = d.User
-		}
 
-		//  s.Password = sudoPass
-		s.Password = v.Password
-		if s.Password == "" {
-			s.Password = d.Password
+		switch {
+		case v.User != "":
+			s.User = v.User
+		case d.User != "":
+			s.User = d.User
+		default:
+			return fmt.Errorf("%s is invalid. User is empty", name)
 		}
 
 		s.Host = v.Host
+		if len(s.Host) == 0 {
+			return fmt.Errorf("%s is invalid. host is empty", name)
+		}
 
-		s.Port = v.Port
-		if s.Port == "" {
+		switch {
+		case v.Port != "":
+			s.Port = v.Port
+		case d.Port != "":
 			s.Port = d.Port
+		default:
+			s.Port = "22"
 		}
 
 		s.KeyPath = v.KeyPath
-		if s.KeyPath == "" {
+		if len(s.KeyPath) == 0 {
 			s.KeyPath = d.KeyPath
 		}
 		if s.KeyPath != "" {
 			if _, err := os.Stat(s.KeyPath); err != nil {
 				return fmt.Errorf(
-					"config.toml is invalid. keypath: %s not exists", s.KeyPath)
+					"%s is invalid. keypath: %s not exists", name, s.KeyPath)
 			}
 		}
 
 		//  s.KeyPassword = keyPass
 		s.KeyPassword = v.KeyPassword
-		if s.KeyPassword == "" {
+		if len(s.KeyPassword) == 0 {
 			s.KeyPassword = d.KeyPassword
 		}
 
@@ -101,13 +107,78 @@ func (c TOMLLoader) Load(pathToToml, keyPass, sudoPass string) (err error) {
 			s.CpeNames = d.CpeNames
 		}
 
+		s.DependencyCheckXMLPath = v.DependencyCheckXMLPath
+		if len(s.DependencyCheckXMLPath) == 0 {
+			s.DependencyCheckXMLPath = d.DependencyCheckXMLPath
+		}
+
+		// Load CPEs from OWASP Dependency Check XML
+		if len(s.DependencyCheckXMLPath) != 0 {
+			cpes, err := parser.Parse(s.DependencyCheckXMLPath)
+			if err != nil {
+				return fmt.Errorf(
+					"Failed to read OWASP Dependency Check XML: %s", err)
+			}
+			log.Infof("Loaded from OWASP Dependency Check XML: %s",
+				s.ServerName)
+			s.CpeNames = append(s.CpeNames, cpes...)
+		}
+
+		s.Containers = v.Containers
+		if len(s.Containers) == 0 {
+			s.Containers = d.Containers
+		}
+
+		s.IgnoreCves = v.IgnoreCves
+		for _, cve := range d.IgnoreCves {
+			found := false
+			for _, c := range s.IgnoreCves {
+				if cve == c {
+					found = true
+					break
+				}
+			}
+			if !found {
+				s.IgnoreCves = append(s.IgnoreCves, cve)
+			}
+		}
+
+		s.Optional = v.Optional
+		for _, dkv := range d.Optional {
+			found := false
+			for _, kv := range s.Optional {
+				if dkv[0] == kv[0] {
+					found = true
+					break
+				}
+			}
+			if !found {
+				s.Optional = append(s.Optional, dkv)
+			}
+		}
+
+		s.Enablerepo = v.Enablerepo
+		if len(s.Enablerepo) == 0 {
+			s.Enablerepo = d.Enablerepo
+		}
+		if len(s.Enablerepo) != 0 {
+			for _, repo := range strings.Split(s.Enablerepo, ",") {
+				switch repo {
+				case "base", "updates":
+					// nop
+				default:
+					return fmt.Errorf(
+						"For now, enablerepo have to be base or updates: %s, servername: %s",
+						s.Enablerepo, name)
+				}
+			}
+		}
+
 		s.LogMsgAnsiColor = Colors[i%len(Colors)]
 		i++
 
 		servers[name] = s
 	}
-	log.Debug("Config loaded")
-	log.Debugf("%s", pp.Sprintf("%v", servers))
 	Conf.Servers = servers
-	return
+	return nil
 }

contrib/owasp-dependency-check/parser/parser.go

@@ -0,0 +1,64 @@
+package parser
+
+import (
+	"encoding/xml"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"sort"
+	"strings"
+)
+
+type analysis struct {
+	Dependencies []dependency `xml:"dependencies>dependency"`
+}
+
+type dependency struct {
+	Identifiers []identifier `xml:"identifiers>identifier"`
+}
+
+type identifier struct {
+	Name string `xml:"name"`
+	Type string `xml:"type,attr"`
+}
+
+func appendIfMissing(slice []string, str string) []string {
+	for _, s := range slice {
+		if s == str {
+			return slice
+		}
+	}
+	return append(slice, str)
+}
+
+// Parse parses XML and collect list of cpe
+func Parse(path string) ([]string, error) {
+	file, err := os.Open(path)
+	if err != nil {
+		return []string{}, fmt.Errorf("Failed to open: %s", err)
+	}
+	defer file.Close()
+
+	b, err := ioutil.ReadAll(file)
+	if err != nil {
+		return []string{}, fmt.Errorf("Failed to read: %s", err)
+	}
+
+	var anal analysis
+	if err := xml.Unmarshal(b, &anal); err != nil {
+		fmt.Errorf("Failed to unmarshal: %s", err)
+	}
+
+	cpes := []string{}
+	for _, d := range anal.Dependencies {
+		for _, ident := range d.Identifiers {
+			if ident.Type == "cpe" {
+				name := strings.TrimPrefix(ident.Name, "(")
+				name = strings.TrimSuffix(name, ")")
+				cpes = appendIfMissing(cpes, name)
+			}
+		}
+	}
+	sort.Strings(cpes)
+	return cpes, nil
+}

cveapi/cve_client.go

@@ -30,6 +30,8 @@ import (
 	log "github.com/Sirupsen/logrus"
 	"github.com/future-architect/vuls/config"
 	"github.com/future-architect/vuls/util"
+	cveconfig "github.com/kotakanbe/go-cve-dictionary/config"
+	cvedb "github.com/kotakanbe/go-cve-dictionary/db"
 	cve "github.com/kotakanbe/go-cve-dictionary/models"
 )
 
@@ -46,17 +48,19 @@ func (api *cvedictClient) initialize() {
 }
 
 func (api cvedictClient) CheckHealth() (ok bool, err error) {
+	if config.Conf.CveDBPath != "" {
+		log.Debugf("get cve-dictionary from %s", config.Conf.CveDBType)
+		return true, nil
+	}
+
 	api.initialize()
 	url := fmt.Sprintf("%s/health", api.baseURL)
 	var errs []error
 	var resp *http.Response
 	resp, _, errs = gorequest.New().SetDebug(config.Conf.Debug).Get(url).End()
 	//  resp, _, errs = gorequest.New().Proxy(api.httpProxy).Get(url).End()
-	if len(errs) > 0 || resp.StatusCode != 200 {
-		return false, fmt.Errorf("Failed to request to CVE server. url: %s, errs: %v",
-			url,
-			errs,
-		)
+	if 0 < len(errs) || resp == nil || resp.StatusCode != 200 {
+		return false, fmt.Errorf("Failed to request to CVE server. url: %s, errs: %v", url, errs)
 	}
 	return true, nil
 }
@@ -67,6 +71,10 @@ type response struct {
 }
 
 func (api cvedictClient) FetchCveDetails(cveIDs []string) (cveDetails cve.CveDetails, err error) {
+	if config.Conf.CveDBPath != "" {
+		return api.FetchCveDetailsFromCveDB(cveIDs)
+	}
+
 	api.baseURL = config.Conf.CveDictionaryURL
 	reqChan := make(chan string, len(cveIDs))
 	resChan := make(chan response, len(cveIDs))
@@ -126,6 +134,31 @@ func (api cvedictClient) FetchCveDetails(cveIDs []string) (cveDetails cve.CveDet
 	return
 }
 
+func (api cvedictClient) FetchCveDetailsFromCveDB(cveIDs []string) (cveDetails cve.CveDetails, err error) {
+	log.Debugf("open cve-dictionary db (%s)", config.Conf.CveDBType)
+	cveconfig.Conf.DBType = config.Conf.CveDBType
+	cveconfig.Conf.DBPath = config.Conf.CveDBPath
+	cveconfig.Conf.DebugSQL = config.Conf.DebugSQL
+	if err := cvedb.OpenDB(); err != nil {
+		return []cve.CveDetail{},
+			fmt.Errorf("Failed to open DB. err: %s", err)
+	}
+	for _, cveID := range cveIDs {
+		cveDetail := cvedb.Get(cveID)
+		if len(cveDetail.CveID) == 0 {
+			cveDetails = append(cveDetails, cve.CveDetail{
+				CveID: cveID,
+			})
+		} else {
+			cveDetails = append(cveDetails, cveDetail)
+		}
+	}
+
+	// order by CVE ID desc
+	sort.Sort(cveDetails)
+	return
+}
+
 func (api cvedictClient) httpGet(key, url string, resChan chan<- response, errChan chan<- error) {
 	var body string
 	var errs []error
@@ -133,8 +166,8 @@ func (api cvedictClient) httpGet(key, url string, resChan chan<- response, errCh
 	f := func() (err error) {
 		//  resp, body, errs = gorequest.New().SetDebug(config.Conf.Debug).Get(url).End()
 		resp, body, errs = gorequest.New().Get(url).End()
-		if len(errs) > 0 || resp.StatusCode != 200 {
-			return fmt.Errorf("HTTP GET error: %v, code: %d, url: %s", errs, resp.StatusCode, url)
+		if 0 < len(errs) || resp == nil || resp.StatusCode != 200 {
+			return fmt.Errorf("HTTP GET error: %v, url: %s, resp: %v", errs, url, resp)
 		}
 		return nil
 	}
@@ -155,49 +188,17 @@ func (api cvedictClient) httpGet(key, url string, resChan chan<- response, errCh
 	}
 }
 
-//  func (api cvedictClient) httpGet(key, url string, query map[string]string, resChan chan<- response, errChan chan<- error) {
-
-//      var body string
-//      var errs []error
-//      var resp *http.Response
-//      f := func() (err error) {
-//          req := gorequest.New().SetDebug(true).Proxy(api.httpProxy).Get(url)
-//          for key := range query {
-//              req = req.Query(fmt.Sprintf("%s=%s", key, query[key])).Set("Content-Type", "application/x-www-form-urlencoded")
-//          }
-//          pp.Println(req)
-//          resp, body, errs = req.End()
-//          if len(errs) > 0 || resp.StatusCode != 200 {
-//              errChan <- fmt.Errorf("HTTP error. errs: %v, url: %s", errs, url)
-//          }
-//          return nil
-//      }
-//      notify := func(err error, t time.Duration) {
-//          log.Warnf("Failed to get. retrying in %s seconds. err: %s", t, err)
-//      }
-//      err := backoff.RetryNotify(f, backoff.NewExponentialBackOff(), notify)
-//      if err != nil {
-//          errChan <- fmt.Errorf("HTTP Error %s", err)
-//      }
-//      //  resChan <- body
-//      cveDetail := cve.CveDetail{}
-//      if err := json.Unmarshal([]byte(body), &cveDetail); err != nil {
-//          errChan <- fmt.Errorf("Failed to Unmarshall. body: %s, err: %s", body, err)
-//      }
-//      resChan <- response{
-//          key,
-//          cveDetail,
-//      }
-//  }
-
 type responseGetCveDetailByCpeName struct {
 	CpeName    string
 	CveDetails []cve.CveDetail
 }
 
 func (api cvedictClient) FetchCveDetailsByCpeName(cpeName string) ([]cve.CveDetail, error) {
-	api.baseURL = config.Conf.CveDictionaryURL
+	if config.Conf.CveDBPath != "" {
+		return api.FetchCveDetailsByCpeNameFromDB(cpeName)
+	}
 
+	api.baseURL = config.Conf.CveDictionaryURL
 	url, err := util.URLPathJoin(api.baseURL, "cpes")
 	if err != nil {
 		return []cve.CveDetail{}, err
@@ -218,8 +219,8 @@ func (api cvedictClient) httpPost(key, url string, query map[string]string) ([]c
 			req = req.Send(fmt.Sprintf("%s=%s", key, query[key])).Type("json")
 		}
 		resp, body, errs = req.End()
-		if len(errs) > 0 || resp.StatusCode != 200 {
-			return fmt.Errorf("HTTP POST errors: %v, code: %d, url: %s", errs, resp.StatusCode, url)
+		if 0 < len(errs) || resp == nil || resp.StatusCode != 200 {
+			return fmt.Errorf("HTTP POST error: %v, url: %s, resp: %v", errs, url, resp)
 		}
 		return nil
 	}
@@ -238,3 +239,16 @@ func (api cvedictClient) httpPost(key, url string, query map[string]string) ([]c
 	}
 	return cveDetails, nil
 }
+
+func (api cvedictClient) FetchCveDetailsByCpeNameFromDB(cpeName string) ([]cve.CveDetail, error) {
+	log.Debugf("open cve-dictionary db (%s)", config.Conf.CveDBType)
+	cveconfig.Conf.DBType = config.Conf.CveDBType
+	cveconfig.Conf.DBPath = config.Conf.CveDBPath
+	cveconfig.Conf.DebugSQL = config.Conf.DebugSQL
+
+	if err := cvedb.OpenDB(); err != nil {
+		return []cve.CveDetail{},
+			fmt.Errorf("Failed to open DB. err: %s", err)
+	}
+	return cvedb.GetByCpeName(cpeName), nil
+}

db/db.go

@@ -1,272 +0,0 @@
-/* Vuls - Vulnerability Scanner
-Copyright (C) 2016  Future Architect, Inc. Japan.
-
-This program is free software: you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation, either version 3 of the License, or
-(at your option) any later version.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program.  If not, see <http://www.gnu.org/licenses/>.
-*/
-
-package db
-
-import (
-	"fmt"
-	"sort"
-	"time"
-
-	"github.com/future-architect/vuls/config"
-	m "github.com/future-architect/vuls/models"
-	"github.com/jinzhu/gorm"
-	cvedb "github.com/kotakanbe/go-cve-dictionary/db"
-	cve "github.com/kotakanbe/go-cve-dictionary/models"
-)
-
-var db *gorm.DB
-
-// OpenDB opens Database
-func OpenDB() (err error) {
-	db, err = gorm.Open("sqlite3", config.Conf.DBPath)
-	if err != nil {
-		err = fmt.Errorf("Failed to open DB. datafile: %s, err: %s", config.Conf.DBPath, err)
-		return
-
-	}
-	db.LogMode(config.Conf.DebugSQL)
-	return
-}
-
-// MigrateDB migrates Database
-func MigrateDB() error {
-	if err := db.AutoMigrate(
-		&m.ScanHistory{},
-		&m.ScanResult{},
-		//  &m.NWLink{},
-		&m.CveInfo{},
-		&m.CpeName{},
-		&m.PackageInfo{},
-		&m.DistroAdvisory{},
-		&cve.CveDetail{},
-		&cve.Jvn{},
-		&cve.Nvd{},
-		&cve.Reference{},
-		&cve.Cpe{},
-	).Error; err != nil {
-		return fmt.Errorf("Failed to migrate. err: %s", err)
-	}
-
-	errMsg := "Failed to create index. err: %s"
-	//  if err := db.Model(&m.NWLink{}).
-	//      AddIndex("idx_n_w_links_scan_result_id", "scan_result_id").Error; err != nil {
-	//      return fmt.Errorf(errMsg, err)
-	//  }
-	if err := db.Model(&m.CveInfo{}).
-		AddIndex("idx_cve_infos_scan_result_id", "scan_result_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&m.CpeName{}).
-		AddIndex("idx_cpe_names_cve_info_id", "cve_info_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&m.PackageInfo{}).
-		AddIndex("idx_package_infos_cve_info_id", "cve_info_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&m.DistroAdvisory{}).
-		//TODO check table name
-		AddIndex("idx_distro_advisories_cve_info_id", "cve_info_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&cve.CveDetail{}).
-		AddIndex("idx_cve_detail_cve_info_id", "cve_info_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&cve.CveDetail{}).
-		AddIndex("idx_cve_detail_cveid", "cve_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&cve.Nvd{}).
-		AddIndex("idx_nvds_cve_detail_id", "cve_detail_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&cve.Jvn{}).
-		AddIndex("idx_jvns_cve_detail_id", "cve_detail_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&cve.Cpe{}).
-		AddIndex("idx_cpes_jvn_id", "jvn_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&cve.Reference{}).
-		AddIndex("idx_references_jvn_id", "jvn_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&cve.Cpe{}).
-		AddIndex("idx_cpes_nvd_id", "nvd_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-	if err := db.Model(&cve.Reference{}).
-		AddIndex("idx_references_nvd_id", "nvd_id").Error; err != nil {
-		return fmt.Errorf(errMsg, err)
-	}
-
-	return nil
-}
-
-// Insert inserts scan results into DB
-func Insert(results []m.ScanResult) error {
-	for _, r := range results {
-		r.KnownCves = resetGormIDs(r.KnownCves)
-		r.UnknownCves = resetGormIDs(r.UnknownCves)
-	}
-
-	history := m.ScanHistory{
-		ScanResults: results,
-		ScannedAt:   time.Now(),
-	}
-
-	db = db.Set("gorm:save_associations", false)
-	if err := db.Create(&history).Error; err != nil {
-		return err
-	}
-	for _, scanResult := range history.ScanResults {
-		scanResult.ScanHistoryID = history.ID
-		if err := db.Create(&scanResult).Error; err != nil {
-			return err
-		}
-		if err := insertCveInfos(scanResult.ID, scanResult.KnownCves); err != nil {
-			return err
-		}
-		if err := insertCveInfos(scanResult.ID, scanResult.UnknownCves); err != nil {
-			return err
-		}
-	}
-	return nil
-}
-
-func insertCveInfos(scanResultID uint, infos []m.CveInfo) error {
-	for _, cveInfo := range infos {
-		cveInfo.ScanResultID = scanResultID
-		if err := db.Create(&cveInfo).Error; err != nil {
-			return err
-		}
-
-		for _, pack := range cveInfo.Packages {
-			pack.CveInfoID = cveInfo.ID
-			if err := db.Create(&pack).Error; err != nil {
-				return err
-			}
-		}
-
-		for _, distroAdvisory := range cveInfo.DistroAdvisories {
-			distroAdvisory.CveInfoID = cveInfo.ID
-			if err := db.Create(&distroAdvisory).Error; err != nil {
-				return err
-			}
-		}
-
-		for _, cpeName := range cveInfo.CpeNames {
-			cpeName.CveInfoID = cveInfo.ID
-			if err := db.Create(&cpeName).Error; err != nil {
-				return err
-			}
-		}
-
-		db = db.Set("gorm:save_associations", true)
-		cveDetail := cveInfo.CveDetail
-		cveDetail.CveInfoID = cveInfo.ID
-		if err := db.Create(&cveDetail).Error; err != nil {
-			return err
-		}
-		db = db.Set("gorm:save_associations", false)
-	}
-	return nil
-}
-
-func resetGormIDs(infos []m.CveInfo) []m.CveInfo {
-	for i := range infos {
-		infos[i].CveDetail.ID = 0
-		// NVD
-		infos[i].CveDetail.Nvd.ID = 0
-		for j := range infos[i].CveDetail.Nvd.Cpes {
-			infos[i].CveDetail.Nvd.Cpes[j].ID = 0
-		}
-		for j := range infos[i].CveDetail.Nvd.References {
-			infos[i].CveDetail.Nvd.References[j].ID = 0
-		}
-
-		// JVN
-		infos[i].CveDetail.Jvn.ID = 0
-		for j := range infos[i].CveDetail.Jvn.Cpes {
-			infos[i].CveDetail.Jvn.Cpes[j].ID = 0
-		}
-		for j := range infos[i].CveDetail.Jvn.References {
-			infos[i].CveDetail.Jvn.References[j].ID = 0
-		}
-
-		//Packages
-		for j := range infos[i].Packages {
-			infos[i].Packages[j].ID = 0
-			infos[i].Packages[j].CveInfoID = 0
-		}
-	}
-	return infos
-}
-
-// SelectLatestScanHistory select latest scan history from DB
-func SelectLatestScanHistory() (m.ScanHistory, error) {
-	scanHistory := m.ScanHistory{}
-	db.Order("scanned_at desc").First(&scanHistory)
-
-	if scanHistory.ID == 0 {
-		return m.ScanHistory{}, fmt.Errorf("No scanHistory records")
-	}
-
-	results := []m.ScanResult{}
-	db.Model(&scanHistory).Related(&results, "ScanResults")
-	scanHistory.ScanResults = results
-
-	for i, r := range results {
-		//  nw := []m.NWLink{}
-		//  db.Model(&r).Related(&nw, "NWLinks")
-		//  scanHistory.ScanResults[i].NWLinks = nw
-
-		knownCves := selectCveInfos(&r, "KnownCves")
-		sort.Sort(m.CveInfos(knownCves))
-		scanHistory.ScanResults[i].KnownCves = knownCves
-	}
-	return scanHistory, nil
-}
-
-func selectCveInfos(result *m.ScanResult, fieldName string) []m.CveInfo {
-	cveInfos := []m.CveInfo{}
-	db.Model(&result).Related(&cveInfos, fieldName)
-
-	for i, cveInfo := range cveInfos {
-		cveDetail := cve.CveDetail{}
-		db.Model(&cveInfo).Related(&cveDetail, "CveDetail")
-		id := cveDetail.CveID
-		filledCveDetail := cvedb.Get(id, db)
-		cveInfos[i].CveDetail = filledCveDetail
-
-		packs := []m.PackageInfo{}
-		db.Model(&cveInfo).Related(&packs, "Packages")
-		cveInfos[i].Packages = packs
-
-		advisories := []m.DistroAdvisory{}
-		db.Model(&cveInfo).Related(&advisories, "DistroAdvisories")
-		cveInfos[i].DistroAdvisories = advisories
-
-		names := []m.CpeName{}
-		db.Model(&cveInfo).Related(&names, "CpeNames")
-		cveInfos[i].CpeNames = names
-	}
-	return cveInfos
-}

docker/Dockerfile

@@ -1,14 +0,0 @@
-FROM golang:1.6
-RUN apt-get update \
-    && apt-get upgrade -y \
-    && apt-get install -y git openssh-client gcc
-WORKDIR /app
-RUN go get github.com/kotakanbe/go-cve-dictionary
-RUN go get github.com/future-architect/vuls
-COPY fetch.sh .
-RUN /bin/bash /app/fetch.sh
-COPY config.toml .
-COPY run.sh .
-ENTRYPOINT ["/bin/bash", "/app/run.sh"]
-COPY id_rsa .
-COPY id_rsa.pub .

docker/README.md

@@ -1,2 +0,0 @@
-# Must do
-* Edit your config.toml to match your infrastructure

docker/config.toml

@@ -1 +0,0 @@
-

docker/fetch.sh

@@ -1,2 +0,0 @@
-#!/bin/bash
- for i in {2002..2016}; do go-cve-dictionary fetchnvd -years $i ; done

docker/id_rsa

@@ -1 +0,0 @@
-

docker/id_rsa.pub

@@ -1 +0,0 @@
-

docker/run.sh

@@ -1,4 +0,0 @@
-#!/bin/bash
-go-cve-dictionary server &
-sleep 2
-vuls scan -config /app/config.toml -report-slack

glide.lock

@@ -0,0 +1,123 @@
+hash: ca64aef6e9e94c7be91f79b88edb847363c8a5bd48da4ad27784e9342c8db6e2
+updated: 2016-11-01T15:05:15.23083077+09:00
+imports:
+- name: github.com/asaskevich/govalidator
+  version: 7b3beb6df3c42abd3509abfc3bcacc0fbfb7c877
+- name: github.com/aws/aws-sdk-go
+  version: 9e5bedb97b1cd85e53fd99209f93fd1a8a9f1df7
+  subpackages:
+  - aws
+  - aws/awserr
+  - aws/awsutil
+  - aws/client
+  - aws/client/metadata
+  - aws/corehandlers
+  - aws/credentials
+  - aws/credentials/ec2rolecreds
+  - aws/credentials/endpointcreds
+  - aws/credentials/stscreds
+  - aws/defaults
+  - aws/ec2metadata
+  - aws/request
+  - aws/session
+  - aws/signer/v4
+  - private/endpoints
+  - private/protocol
+  - private/protocol/query
+  - private/protocol/query/queryutil
+  - private/protocol/rest
+  - private/protocol/restxml
+  - private/protocol/xml/xmlutil
+  - private/waiter
+  - service/s3
+  - service/sts
+- name: github.com/Azure/azure-sdk-for-go
+  version: 9016164015faa51e549605e7b4b117f7de2aa6f9
+  subpackages:
+  - storage
+- name: github.com/boltdb/bolt
+  version: 074dffcc83e9f421e261526d297cd93f22a34080
+- name: github.com/BurntSushi/toml
+  version: 99064174e013895bbd9b025c31100bd1d9b590ca
+- name: github.com/cenkalti/backoff
+  version: b02f2bbce11d7ea6b97f282ef1771b0fe2f65ef3
+- name: github.com/cheggaaa/pb
+  version: ad4efe000aa550bb54918c06ebbadc0ff17687b9
+- name: github.com/go-ini/ini
+  version: 6e4869b434bd001f6983749881c7ead3545887d8
+- name: github.com/go-sql-driver/mysql
+  version: 2a6c6079c7eff49a7e9d641e109d922f124a3e4c
+- name: github.com/google/subcommands
+  version: a71b91e238406bd68766ee52db63bebedce0e9f6
+- name: github.com/gosuri/uitable
+  version: 36ee7e946282a3fb1cfecd476ddc9b35d8847e42
+  subpackages:
+  - util/strutil
+  - util/wordwrap
+- name: github.com/howeyc/gopass
+  version: f5387c492211eb133053880d23dfae62aa14123d
+- name: github.com/jinzhu/gorm
+  version: c1b9cf186e4bcd8e5d566ef43f2ae2dfe22dc34e
+  subpackages:
+  - dialects/mysql
+- name: github.com/jinzhu/inflection
+  version: 74387dc39a75e970e7a3ae6a3386b5bd2e5c5cff
+- name: github.com/jmespath/go-jmespath
+  version: bd40a432e4c76585ef6b72d3fd96fb9b6dc7b68d
+- name: github.com/jroimartin/gocui
+  version: 357a541add9e311f7b67dfbaf92e28c71680a6b7
+- name: github.com/k0kubun/pp
+  version: f5dce6ed0ccf6c350f1679964ff6b61f3d6d2033
+- name: github.com/kotakanbe/go-cve-dictionary
+  version: 70989b6709c3102924ad8c8483e9bdc99bcb598b
+  subpackages:
+  - config
+  - db
+  - jvn
+  - log
+  - models
+  - nvd
+  - util
+- name: github.com/kotakanbe/go-pingscanner
+  version: 58e188a3e4f6ab1a6371e33421e4502e26fa1e80
+- name: github.com/kotakanbe/logrus-prefixed-formatter
+  version: f4f7d41649cf1e75e736884da8d05324aa76ea25
+- name: github.com/mattn/go-colorable
+  version: 6c903ff4aa50920ca86087a280590b36b3152b9c
+- name: github.com/mattn/go-isatty
+  version: 66b8e73f3f5cda9f96b69efd03dd3d7fc4a5cdb8
+- name: github.com/mattn/go-runewidth
+  version: 737072b4e32b7a5018b4a7125da8d12de90e8045
+- name: github.com/mattn/go-sqlite3
+  version: e5a3c16c5c1d80b24f633e68aecd6b0702786d3d
+- name: github.com/mgutz/ansi
+  version: c286dcecd19ff979eeb73ea444e479b903f2cfcb
+- name: github.com/moul/http2curl
+  version: c984a4ec331f8ef0e5cd782975a97c92bd8ab40c
+- name: github.com/nsf/termbox-go
+  version: b6acae516ace002cb8105a89024544a1480655a5
+- name: github.com/parnurzeal/gorequest
+  version: e37b9d1efacf7c94820b29b75dd7d0c2996b3fb1
+- name: github.com/rifflock/lfshook
+  version: 3f9d976bd7402de39b46357069fb6325a974572e
+- name: github.com/Sirupsen/logrus
+  version: 3ec0642a7fb6488f65b06f9040adc67e3990296a
+- name: golang.org/x/crypto
+  version: 1150b8bd09e53aea1d415621adae9bad665061a1
+  subpackages:
+  - curve25519
+  - ed25519
+  - ed25519/internal/edwards25519
+  - ssh
+  - ssh/agent
+  - ssh/terminal
+- name: golang.org/x/net
+  version: 65dfc08770ce66f74becfdff5f8ab01caef4e946
+  subpackages:
+  - context
+  - publicsuffix
+- name: golang.org/x/sys
+  version: c200b10b5d5e122be351b67af224adc6128af5bf
+  subpackages:
+  - unix
+testImports: []

glide.yaml

@@ -0,0 +1,39 @@
+package: github.com/future-architect/vuls
+import:
+- package: github.com/Azure/azure-sdk-for-go
+  subpackages:
+  - storage
+- package: github.com/BurntSushi/toml
+- package: github.com/Sirupsen/logrus
+- package: github.com/asaskevich/govalidator
+- package: github.com/aws/aws-sdk-go
+  subpackages:
+  - aws
+  - aws/credentials
+  - aws/session
+  - service/s3
+- package: github.com/boltdb/bolt
+- package: github.com/cenkalti/backoff
+- package: github.com/google/subcommands
+- package: github.com/gosuri/uitable
+- package: github.com/howeyc/gopass
+- package: github.com/jinzhu/gorm
+- package: github.com/jroimartin/gocui
+- package: github.com/k0kubun/pp
+- package: github.com/kotakanbe/go-cve-dictionary
+  subpackages:
+  - config
+  - db
+  - models
+- package: github.com/kotakanbe/go-pingscanner
+- package: github.com/kotakanbe/logrus-prefixed-formatter
+- package: github.com/mattn/go-sqlite3
+- package: github.com/parnurzeal/gorequest
+- package: github.com/rifflock/lfshook
+- package: golang.org/x/crypto
+  subpackages:
+  - ssh
+  - ssh/agent
+- package: golang.org/x/net
+  subpackages:
+  - context

img/vuls-scan-flow.graphml

@@ -0,0 +1,417 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<graphml xmlns="http://graphml.graphdrawing.org/xmlns" xmlns:java="http://www.yworks.com/xml/yfiles-common/1.0/java" xmlns:sys="http://www.yworks.com/xml/yfiles-common/markup/primitives/2.0" xmlns:x="http://www.yworks.com/xml/yfiles-common/markup/2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:y="http://www.yworks.com/xml/graphml" xmlns:yed="http://www.yworks.com/xml/yed/3" xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns http://www.yworks.com/xml/schema/graphml/1.1/ygraphml.xsd">
+  <!--Created by yEd 3.14.2-->
+  <key attr.name="Description" attr.type="string" for="graph" id="d0"/>
+  <key for="port" id="d1" yfiles.type="portgraphics"/>
+  <key for="port" id="d2" yfiles.type="portgeometry"/>
+  <key for="port" id="d3" yfiles.type="portuserdata"/>
+  <key attr.name="url" attr.type="string" for="node" id="d4"/>
+  <key attr.name="description" attr.type="string" for="node" id="d5"/>
+  <key for="node" id="d6" yfiles.type="nodegraphics"/>
+  <key for="graphml" id="d7" yfiles.type="resources"/>
+  <key attr.name="url" attr.type="string" for="edge" id="d8"/>
+  <key attr.name="description" attr.type="string" for="edge" id="d9"/>
+  <key for="edge" id="d10" yfiles.type="edgegraphics"/>
+  <graph edgedefault="directed" id="G">
+    <data key="d0"/>
+    <node id="n0">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.process">
+          <y:Geometry height="56.0" width="268.0" x="309.6849206349206" y="0.0"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="18.1328125" modelName="custom" textColor="#000000" visible="true" width="83.482421875" x="92.2587890625" y="18.93359375">Detect the OS<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.5" nodeRatioX="0.0" nodeRatioY="0.1619001116071429" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n1">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.decision">
+          <y:Geometry height="40.0" width="80.0" x="403.6849206349206" y="206.44247787610618"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" hasText="false" height="4.0" modelName="custom" textColor="#000000" visible="true" width="4.0" x="38.0" y="18.0">
+            <y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n2">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.process">
+          <y:Geometry height="90.44247787610618" width="268.0" x="309.6849206349206" y="86.0"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="60.53125" modelName="custom" textColor="#000000" visible="true" width="170.763671875" x="48.61816406250006" y="14.95561393805309">Get installed packages
+Debian/Ubuntu: dpkg-query
+Amazon/RHEL/CentOS: rpm
+FreeBSD: pkg<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="2.220446049250313E-16" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n3">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.process">
+          <y:Geometry height="56.0" width="268.0" x="10.0" y="287.8409153761062"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="32.265625" modelName="custom" textColor="#000000" visible="true" width="260.83984375" x="3.580078125" y="11.8671875">Check upgradable packages
+Debian/Ubuntu: apt-get upgrade --dry-run<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n4">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.loopLimit">
+          <y:Geometry height="51.10998735777497" width="137.19216182048035" x="75.40391908975982" y="376.28592169721867"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="32.265625" modelName="custom" textColor="#000000" visible="true" width="131.751953125" x="2.7201043477401754" y="9.422181178887513">foreach 
+upgradable  packages<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="5.551115123125783E-16" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n5">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.process">
+          <y:Geometry height="56.0" width="268.0" x="10.0" y="459.8409153761062"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="32.265625" modelName="custom" textColor="#000000" visible="true" width="213.619140625" x="27.1904296875" y="11.8671875">Parse changelog and get  CVE IDs
+Debian/Ubuntu: aptitude changelog<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n6">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.loopLimitEnd">
+          <y:Geometry height="50.0" width="137.0" x="75.5" y="545.8409153761062"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="18.1328125" modelName="custom" textColor="#000000" visible="true" width="55.24609375" x="40.876953125" y="15.93359375">end loop<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n7">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.process">
+          <y:Geometry height="56.0" width="268.0" x="609.3698412698412" y="625.8409153761062"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="18.1328125" modelName="custom" textColor="#000000" visible="true" width="194.904296875" x="36.5478515625" y="18.93359375">Select the CVE detail information<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.5" nodeRatioX="0.0" nodeRatioY="0.1619001116071429" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n8">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.process">
+          <y:Geometry height="56.0" width="268.0" x="609.3698412698412" y="287.8409153761062"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="46.3984375" modelName="custom" textColor="#000000" visible="true" width="232.744140625" x="17.6279296875" y="4.80078125">Get CVE IDs by using package manager
+Amazon/RHEL: yum plugin security
+FreeBSD: pkg audit<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n9">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.dataBase">
+          <y:Geometry height="65.22882427307195" width="136.83944374209864" x="411.5802781289507" y="687.385587863464"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="18.1328125" modelName="custom" textColor="#000000" visible="true" width="117.970703125" x="9.434370308549205" y="23.548005886535975">CVE DB (NVD / JVN)<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="-8.326672684688674E-16" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n10">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.process">
+          <y:Geometry height="56.0" width="268.0" x="609.3698412698412" y="716.4553275126422"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="32.265625" modelName="custom" textColor="#000000" visible="true" width="152.634765625" x="57.6826171875" y="11.8671875">Write results to JSON files
+Reporting<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n11">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.process">
+          <y:Geometry height="56.0" width="268.0" x="309.6849206349206" y="287.8409153761062"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="32.265625" modelName="custom" textColor="#000000" visible="true" width="293.06640625" x="-12.533203124999943" y="11.8671875">Get all changelogs of updatable packages at once
+CentOS: yum update --changelog<y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="2.220446049250313E-16" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <node id="n12">
+      <data key="d6">
+        <y:GenericNode configuration="com.yworks.flowchart.process">
+          <y:Geometry height="56.0" width="268.0" x="309.6849206349206" y="373.8409153761062"/>
+          <y:Fill color="#E8EEF7" color2="#B7C9E3" transparent="false"/>
+          <y:BorderStyle color="#000000" type="line" width="1.0"/>
+          <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="18.1328125" modelName="custom" textColor="#000000" visible="true" width="205.52734375" x="31.236328125000057" y="18.93359375">Parse changelogs and get CVE IDs <y:LabelModel>
+              <y:SmartNodeLabelModel distance="4.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.5" nodeRatioX="2.220446049250313E-16" nodeRatioY="0.1619001116071429" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/>
+            </y:ModelParameter>
+          </y:NodeLabel>
+        </y:GenericNode>
+      </data>
+    </node>
+    <edge id="e0" source="n2" target="n1">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="0.0" sy="45.22123893805309" tx="0.0" ty="-20.0"/>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="none"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e1" source="n1" target="n3">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="-40.0" sy="0.0" tx="0.0" ty="-28.0">
+            <y:Point x="144.0" y="226.44247787610618"/>
+          </y:Path>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:EdgeLabel alignment="center" configuration="AutoFlippingLabel" distance="2.0" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="32.265625" modelName="custom" preferredPlacement="anywhere" ratio="0.5" textColor="#000000" visible="true" width="46.697265625" x="-56.79057374984495" y="-34.26562148912808">Debian
+Ubuntu<y:LabelModel>
+              <y:SmartEdgeLabelModel autoRotationEnabled="false" defaultAngle="0.0" defaultDistance="10.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartEdgeLabelModelParameter angle="6.283185307179586" distance="1.9999999999998863" distanceToCenter="false" position="right" ratio="0.02215389573439544" segment="0"/>
+            </y:ModelParameter>
+            <y:PreferredPlacementDescriptor angle="0.0" angleOffsetOnRightSide="0" angleReference="absolute" angleRotationOnRightSide="co" distance="-1.0" frozen="true" placement="anywhere" side="anywhere" sideReference="relative_to_edge_flow"/>
+          </y:EdgeLabel>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e2" source="n3" target="n4">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="0.0" sy="28.0" tx="0.0" ty="-25.554993678887485"/>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e3" source="n4" target="n5">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="0.0" sy="25.554993678887485" tx="0.0" ty="-28.0"/>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e4" source="n5" target="n6">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="0.0" sy="28.0" tx="0.0" ty="-25.0"/>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e5" source="n6" target="n7">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="68.5" sy="0.0" tx="0.0" ty="-28.0">
+            <y:Point x="743.3698412698412" y="570.8409153761062"/>
+          </y:Path>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e6" source="n1" target="n8">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="40.0" sy="0.0" tx="0.0" ty="-28.0">
+            <y:Point x="743.3698412698412" y="226.44247787610618"/>
+          </y:Path>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:EdgeLabel alignment="center" configuration="AutoFlippingLabel" distance="2.0" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="46.3984375" modelName="custom" preferredPlacement="anywhere" ratio="0.5" textColor="#000000" visible="true" width="51.806640625" x="10.125014629061297" y="-48.39843398912805">Amazon
+RHEL
+FreeBSD<y:LabelModel>
+              <y:SmartEdgeLabelModel autoRotationEnabled="false" defaultAngle="0.0" defaultDistance="10.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartEdgeLabelModelParameter angle="6.283185307179586" distance="1.9999999999998863" distanceToCenter="false" position="left" ratio="0.022401276994204813" segment="0"/>
+            </y:ModelParameter>
+            <y:PreferredPlacementDescriptor angle="0.0" angleOffsetOnRightSide="0" angleReference="absolute" angleRotationOnRightSide="co" distance="-1.0" frozen="true" placement="anywhere" side="anywhere" sideReference="relative_to_edge_flow"/>
+          </y:EdgeLabel>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e7" source="n8" target="n7">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="0.0" sy="28.0" tx="0.0" ty="-28.0"/>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e8" source="n0" target="n2">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="0.0" sy="28.0" tx="0.0" ty="-45.22123893805309"/>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e9" source="n7" target="n10">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="0.0" sy="28.0" tx="0.0" ty="-28.0"/>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e10" source="n7" target="n9">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="-134.01566143419018" sy="6.159084623893818" tx="0.0" ty="-29.333162136535975">
+            <y:Point x="480.0" y="660.0"/>
+          </y:Path>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="none"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e11" source="n1" target="n11">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="0.0" sy="20.0" tx="0.0" ty="-28.0"/>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:EdgeLabel alignment="center" configuration="AutoFlippingLabel" distance="2.0" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="18.1328125" modelName="custom" preferredPlacement="anywhere" ratio="0.5" textColor="#000000" visible="true" width="46.708984375" x="-53.35447755843876" y="11.632816010871807">CentOS<y:LabelModel>
+              <y:SmartEdgeLabelModel autoRotationEnabled="false" defaultAngle="0.0" defaultDistance="10.0"/>
+            </y:LabelModel>
+            <y:ModelParameter>
+              <y:SmartEdgeLabelModelParameter angle="0.0" distance="30.0" distanceToCenter="true" position="right" ratio="0.5" segment="0"/>
+            </y:ModelParameter>
+            <y:PreferredPlacementDescriptor angle="0.0" angleOffsetOnRightSide="0" angleReference="absolute" angleRotationOnRightSide="co" distance="-1.0" frozen="true" placement="anywhere" side="anywhere" sideReference="relative_to_edge_flow"/>
+          </y:EdgeLabel>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e12" source="n11" target="n12">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="0.0" sy="28.0" tx="0.0" ty="-28.0"/>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+    <edge id="e13" source="n12" target="n7">
+      <data key="d10">
+        <y:PolyLineEdge>
+          <y:Path sx="134.00000000000006" sy="0.0" tx="0.0" ty="-28.0">
+            <y:Point x="743.3698412698412" y="401.8409153761062"/>
+          </y:Path>
+          <y:LineStyle color="#000000" type="line" width="1.0"/>
+          <y:Arrows source="none" target="standard"/>
+          <y:BendStyle smoothed="false"/>
+        </y:PolyLineEdge>
+      </data>
+    </edge>
+  </graph>
+  <data key="d7">
+    <y:Resources/>
+  </data>
+</graphml>

main.go

@@ -19,9 +19,10 @@ package main
 
 import (
 	"flag"
+	"fmt"
 	"os"
 
-	"golang.org/x/net/context"
+	"context"
 
 	"github.com/future-architect/vuls/commands"
 	"github.com/google/subcommands"
@@ -29,6 +30,12 @@ import (
 	_ "github.com/mattn/go-sqlite3"
 )
 
+// Version of Vuls
+var version = "0.1.7"
+
+// Revision of Git
+var revision string
+
 func main() {
 	subcommands.Register(subcommands.HelpCommand(), "")
 	subcommands.Register(subcommands.FlagsCommand(), "")
@@ -37,8 +44,18 @@ func main() {
 	subcommands.Register(&commands.TuiCmd{}, "tui")
 	subcommands.Register(&commands.ScanCmd{}, "scan")
 	subcommands.Register(&commands.PrepareCmd{}, "prepare")
+	subcommands.Register(&commands.HistoryCmd{}, "history")
+	subcommands.Register(&commands.ConfigtestCmd{}, "configtest")
+
+	var v = flag.Bool("v", false, "Show version")
 
 	flag.Parse()
+
+	if *v {
+		fmt.Printf("vuls %s %s\n", version, revision)
+		os.Exit(int(subcommands.ExitSuccess))
+	}
+
 	ctx := context.Background()
 	os.Exit(int(subcommands.Execute(ctx)))
 }

models/models.go

@@ -30,16 +30,34 @@ import (
 // ScanHistory is the history of Scanning.
 type ScanHistory struct {
 	gorm.Model
-	ScanResults []ScanResult
+	ScanResults ScanResults
 	ScannedAt   time.Time
 }
 
 // ScanResults is slice of ScanResult.
 type ScanResults []ScanResult
 
+// Len implement Sort Interface
+func (s ScanResults) Len() int {
+	return len(s)
+}
+
+// Swap implement Sort Interface
+func (s ScanResults) Swap(i, j int) {
+	s[i], s[j] = s[j], s[i]
+}
+
+// Less implement Sort Interface
+func (s ScanResults) Less(i, j int) bool {
+	if s[i].ServerName == s[j].ServerName {
+		return s[i].Container.ContainerID < s[i].Container.ContainerID
+	}
+	return s[i].ServerName < s[j].ServerName
+}
+
 // FilterByCvssOver is filter function.
-func (results ScanResults) FilterByCvssOver() (filtered ScanResults) {
-	for _, result := range results {
+func (s ScanResults) FilterByCvssOver() (filtered ScanResults) {
+	for _, result := range s {
 		cveInfos := []CveInfo{}
 		for _, cveInfo := range result.KnownCves {
 			if config.Conf.CvssScoreOver < cveInfo.CveDetail.CvssScore(config.Conf.Lang) {
@@ -54,17 +72,71 @@ func (results ScanResults) FilterByCvssOver() (filtered ScanResults) {
 
 // ScanResult has the result of scanned CVE information.
 type ScanResult struct {
-	gorm.Model
-	ScanHistoryID uint
+	gorm.Model    `json:"-" xml:"-"`
+	ScanHistoryID uint `json:"-" xml:"-"`
+	ScannedAt     time.Time
 
 	ServerName string // TOML Section key
 	//  Hostname    string
 	Family  string
 	Release string
+
+	Container Container
+
+	Platform Platform
+
 	//  Fqdn        string
 	//  NWLinks     []NWLink
 	KnownCves   []CveInfo
 	UnknownCves []CveInfo
+	IgnoredCves []CveInfo
+
+	Optional [][]interface{} `gorm:"-"`
+}
+
+// ServerInfo returns server name one line
+func (r ScanResult) ServerInfo() string {
+	hostinfo := ""
+	if len(r.Container.ContainerID) == 0 {
+		hostinfo = fmt.Sprintf(
+			"%s (%s%s)",
+			r.ServerName,
+			r.Family,
+			r.Release,
+		)
+	} else {
+		hostinfo = fmt.Sprintf(
+			"%s / %s (%s%s) on %s",
+			r.Container.Name,
+			r.Container.ContainerID,
+			r.Family,
+			r.Release,
+			r.ServerName,
+		)
+	}
+	return hostinfo
+}
+
+// ServerInfoTui returns server infromation for TUI sidebar
+func (r ScanResult) ServerInfoTui() string {
+	hostinfo := ""
+	if len(r.Container.ContainerID) == 0 {
+		hostinfo = fmt.Sprintf(
+			"%s (%s%s)",
+			r.ServerName,
+			r.Family,
+			r.Release,
+		)
+	} else {
+		hostinfo = fmt.Sprintf(
+			"|-- %s (%s%s)",
+			r.Container.Name,
+			r.Family,
+			r.Release,
+			//  r.Container.ContainerID,
+		)
+	}
+	return hostinfo
 }
 
 // CveSummary summarize the number of CVEs group by CVSSv2 Severity
@@ -84,16 +156,24 @@ func (r ScanResult) CveSummary() string {
 			unknown++
 		}
 	}
+
+	if config.Conf.IgnoreUnscoredCves {
+		return fmt.Sprintf("Total: %d (High:%d Middle:%d Low:%d)",
+			high+middle+low, high, middle, low)
+	}
 	return fmt.Sprintf("Total: %d (High:%d Middle:%d Low:%d ?:%d)",
-		high+middle+low+unknown,
-		high, middle, low, unknown,
-	)
+		high+middle+low+unknown, high, middle, low, unknown)
+}
+
+// AllCves returns Known and Unknown CVEs
+func (r ScanResult) AllCves() []CveInfo {
+	return append(r.KnownCves, r.UnknownCves...)
 }
 
 // NWLink has network link information.
 type NWLink struct {
-	gorm.Model
-	ScanResultID uint
+	gorm.Model   `json:"-" xml:"-"`
+	ScanResultID uint `json:"-" xml:"-"`
 
 	IPAddress string
 	Netmask   string
@@ -114,13 +194,16 @@ func (c CveInfos) Swap(i, j int) {
 
 func (c CveInfos) Less(i, j int) bool {
 	lang := config.Conf.Lang
-	return c[i].CveDetail.CvssScore(lang) > c[j].CveDetail.CvssScore(lang)
+	if c[i].CveDetail.CvssScore(lang) == c[j].CveDetail.CvssScore(lang) {
+		return c[i].CveDetail.CveID < c[j].CveDetail.CveID
+	}
+	return c[j].CveDetail.CvssScore(lang) < c[i].CveDetail.CvssScore(lang)
 }
 
 // CveInfo has Cve Information.
 type CveInfo struct {
-	gorm.Model
-	ScanResultID uint
+	gorm.Model   `json:"-" xml:"-"`
+	ScanResultID uint `json:"-" xml:"-"`
 
 	CveDetail        cve.CveDetail
 	Packages         []PackageInfo
@@ -130,8 +213,8 @@ type CveInfo struct {
 
 // CpeName has CPE name
 type CpeName struct {
-	gorm.Model
-	CveInfoID uint
+	gorm.Model `json:"-" xml:"-"`
+	CveInfoID  uint `json:"-" xml:"-"`
 
 	Name string
 }
@@ -196,15 +279,15 @@ func (ps PackageInfoList) FindByName(name string) (result PackageInfo, found boo
 
 // PackageInfo has installed packages.
 type PackageInfo struct {
-	gorm.Model
-	CveInfoID uint
-
-	Name    string
-	Version string
-	Release string
+	gorm.Model `json:"-" xml:"-"`
+	CveInfoID  uint `json:"-" xml:"-"`
 
+	Name       string
+	Version    string
+	Release    string
 	NewVersion string
 	NewRelease string
+	Repository string
 }
 
 // ToStringCurrentVersion returns package name-version-release
@@ -231,14 +314,31 @@ func (p PackageInfo) ToStringNewVersion() string {
 	return str
 }
 
-// DistroAdvisory has Amazon Linux AMI Security Advisory information.
-//TODO Rename to DistroAdvisory
+// DistroAdvisory has Amazon Linux, RHEL, FreeBSD Security Advisory information.
 type DistroAdvisory struct {
-	gorm.Model
-	CveInfoID uint
+	gorm.Model `json:"-" xml:"-"`
+	CveInfoID  uint `json:"-" xml:"-"`
 
 	AdvisoryID string
 	Severity   string
 	Issued     time.Time
 	Updated    time.Time
 }
+
+// Container has Container information
+type Container struct {
+	gorm.Model   `json:"-" xml:"-"`
+	ScanResultID uint `json:"-" xml:"-"`
+
+	ContainerID string
+	Name        string
+}
+
+// Platform has platform information
+type Platform struct {
+	gorm.Model   `json:"-" xml:"-"`
+	ScanResultID uint `json:"-" xml:"-"`
+
+	Name       string // aws or azure or gcp or other...
+	InstanceID string
+}

report/azureblob.go

@@ -0,0 +1,140 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package report
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"time"
+
+	"github.com/Azure/azure-sdk-for-go/storage"
+
+	c "github.com/future-architect/vuls/config"
+	"github.com/future-architect/vuls/models"
+	"github.com/future-architect/vuls/util"
+)
+
+// AzureBlobWriter writes results to AzureBlob
+type AzureBlobWriter struct{}
+
+// CheckIfAzureContainerExists check the existence of Azure storage container
+func CheckIfAzureContainerExists() error {
+	cli, err := getBlobClient()
+	if err != nil {
+		return err
+	}
+	ok, err := cli.ContainerExists(c.Conf.AzureContainer)
+	if err != nil {
+		return err
+	}
+	if !ok {
+		return fmt.Errorf("Container not found. Container: %s", c.Conf.AzureContainer)
+	}
+	return nil
+}
+
+func getBlobClient() (storage.BlobStorageClient, error) {
+	api, err := storage.NewBasicClient(c.Conf.AzureAccount, c.Conf.AzureKey)
+	if err != nil {
+		return storage.BlobStorageClient{}, err
+	}
+	return api.GetBlobService(), nil
+}
+
+// Write results to Azure Blob storage
+func (w AzureBlobWriter) Write(scanResults []models.ScanResult) (err error) {
+	reqChan := make(chan models.ScanResult, len(scanResults))
+	resChan := make(chan bool)
+	errChan := make(chan error, len(scanResults))
+	defer close(resChan)
+	defer close(errChan)
+	defer close(reqChan)
+
+	timeout := time.After(10 * 60 * time.Second)
+	concurrency := 10
+	tasks := util.GenWorkers(concurrency)
+
+	go func() {
+		for _, r := range scanResults {
+			reqChan <- r
+		}
+	}()
+
+	for range scanResults {
+		tasks <- func() {
+			select {
+			case sresult := <-reqChan:
+				func(r models.ScanResult) {
+					err := w.upload(r)
+					if err != nil {
+						errChan <- err
+					}
+					resChan <- true
+				}(sresult)
+			}
+		}
+	}
+
+	errs := []error{}
+	for i := 0; i < len(scanResults); i++ {
+		select {
+		case <-resChan:
+		case err := <-errChan:
+			errs = append(errs, err)
+		case <-timeout:
+			errs = append(errs, fmt.Errorf("Timeout while uploading to azure Blob"))
+		}
+	}
+
+	if 0 < len(errs) {
+		return fmt.Errorf("Failed to upload json to Azure Blob: %v", errs)
+	}
+	return nil
+}
+
+func (w AzureBlobWriter) upload(res models.ScanResult) (err error) {
+	cli, err := getBlobClient()
+	if err != nil {
+		return err
+	}
+	timestr := time.Now().Format("20060102_1504")
+	name := ""
+	if len(res.Container.ContainerID) == 0 {
+		name = fmt.Sprintf("%s/%s.json", timestr, res.ServerName)
+	} else {
+		name = fmt.Sprintf("%s/%s_%s.json", timestr, res.ServerName, res.Container.Name)
+	}
+
+	jsonBytes, err := json.Marshal(res)
+	if err != nil {
+		return fmt.Errorf("Failed to Marshal to JSON: %s", err)
+	}
+
+	if err = cli.CreateBlockBlobFromReader(
+		c.Conf.AzureContainer,
+		name,
+		uint64(len(jsonBytes)),
+		bytes.NewReader(jsonBytes),
+		map[string]string{},
+	); err != nil {
+		return fmt.Errorf("%s/%s, %s",
+			c.Conf.AzureContainer, name, err)
+	}
+	return
+}

report/json.go

@@ -20,18 +20,132 @@ package report
 import (
 	"encoding/json"
 	"fmt"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"regexp"
+	"sort"
+	"strings"
+	"time"
 
+	c "github.com/future-architect/vuls/config"
 	"github.com/future-architect/vuls/models"
 )
 
-// JSONWriter writes report as JSON format
-type JSONWriter struct{}
+// JSONDirs array of json files path.
+type JSONDirs []string
+
+func (d JSONDirs) Len() int {
+	return len(d)
+}
+func (d JSONDirs) Swap(i, j int) {
+	d[i], d[j] = d[j], d[i]
+}
+func (d JSONDirs) Less(i, j int) bool {
+	return d[j] < d[i]
+}
+
+// JSONWriter writes results to file.
+type JSONWriter struct {
+	ScannedAt time.Time
+}
 
 func (w JSONWriter) Write(scanResults []models.ScanResult) (err error) {
-	var j []byte
-	if j, err = json.MarshalIndent(scanResults, "", "  "); err != nil {
-		return
+	var path string
+	if path, err = ensureResultDir(w.ScannedAt); err != nil {
+		return fmt.Errorf("Failed to make direcotory/symlink : %s", err)
+	}
+
+	for _, scanResult := range scanResults {
+		scanResult.ScannedAt = w.ScannedAt
+	}
+
+	var jsonBytes []byte
+	for _, r := range scanResults {
+		jsonPath := ""
+		if len(r.Container.ContainerID) == 0 {
+			jsonPath = filepath.Join(path, fmt.Sprintf("%s.json", r.ServerName))
+		} else {
+			jsonPath = filepath.Join(path,
+				fmt.Sprintf("%s_%s.json", r.ServerName, r.Container.Name))
+		}
+
+		if jsonBytes, err = json.Marshal(r); err != nil {
+			return fmt.Errorf("Failed to Marshal to JSON: %s", err)
+		}
+		if err := ioutil.WriteFile(jsonPath, jsonBytes, 0600); err != nil {
+			return fmt.Errorf("Failed to write JSON. path: %s, err: %s", jsonPath, err)
+		}
 	}
-	fmt.Println(string(j))
 	return nil
 }
+
+// JSONDirPattern is file name pattern of JSON directory
+var JSONDirPattern = regexp.MustCompile(`^\d{8}_\d{4}$`)
+
+// GetValidJSONDirs return valid json directory as array
+func GetValidJSONDirs() (jsonDirs JSONDirs, err error) {
+	var dirInfo []os.FileInfo
+	if dirInfo, err = ioutil.ReadDir(c.Conf.ResultsDir); err != nil {
+		err = fmt.Errorf("Failed to read %s: %s", c.Conf.ResultsDir, err)
+		return
+	}
+	for _, d := range dirInfo {
+		if d.IsDir() && JSONDirPattern.MatchString(d.Name()) {
+			jsonDir := filepath.Join(c.Conf.ResultsDir, d.Name())
+			jsonDirs = append(jsonDirs, jsonDir)
+		}
+	}
+	sort.Sort(jsonDirs)
+	return
+}
+
+// LoadOneScanHistory read JSON data
+func LoadOneScanHistory(jsonDir string) (scanHistory models.ScanHistory, err error) {
+	var scanResults []models.ScanResult
+	var files []os.FileInfo
+	if files, err = ioutil.ReadDir(jsonDir); err != nil {
+		err = fmt.Errorf("Failed to read %s: %s", jsonDir, err)
+		return
+	}
+	for _, file := range files {
+		if filepath.Ext(file.Name()) != ".json" {
+			continue
+		}
+		var scanResult models.ScanResult
+		var data []byte
+		jsonPath := filepath.Join(jsonDir, file.Name())
+		if data, err = ioutil.ReadFile(jsonPath); err != nil {
+			err = fmt.Errorf("Failed to read %s: %s", jsonPath, err)
+			return
+		}
+		if json.Unmarshal(data, &scanResult) != nil {
+			err = fmt.Errorf("Failed to parse %s: %s", jsonPath, err)
+			return
+		}
+		scanResults = append(scanResults, scanResult)
+	}
+	if len(scanResults) == 0 {
+		err = fmt.Errorf("There is no json file under %s", jsonDir)
+		return
+	}
+
+	var scannedAt time.Time
+	if scanResults[0].ScannedAt.IsZero() {
+		splitPath := strings.Split(jsonDir, string(os.PathSeparator))
+		timeStr := splitPath[len(splitPath)-1]
+		timeformat := "20060102_1504"
+		if scannedAt, err = time.Parse(timeformat, timeStr); err != nil {
+			err = fmt.Errorf("Failed to parse %s: %s", timeStr, err)
+			return
+		}
+	} else {
+		scannedAt = scanResults[0].ScannedAt
+	}
+
+	scanHistory = models.ScanHistory{
+		ScanResults: scanResults,
+		ScannedAt:   scannedAt,
+	}
+	return
+}

report/logrus.go

@@ -19,6 +19,8 @@ package report
 
 import (
 	"os"
+	"path/filepath"
+	"runtime"
 
 	"github.com/Sirupsen/logrus"
 	"github.com/future-architect/vuls/models"
@@ -31,7 +33,10 @@ type LogrusWriter struct {
 
 func (w LogrusWriter) Write(scanResults []models.ScanResult) error {
 	path := "/var/log/vuls/report.log"
-	f, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
+	if runtime.GOOS == "windows" {
+		path = filepath.Join(os.Getenv("APPDATA"), "vuls", "report.log")
+	}
+	f, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0600)
 	if err != nil {
 		return err
 	}

report/mail.go

@@ -18,13 +18,14 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 package report
 
 import (
-	"crypto/tls"
 	"fmt"
-	"strconv"
+	"net"
+	"net/mail"
+	"net/smtp"
+	"strings"
 
 	"github.com/future-architect/vuls/config"
 	"github.com/future-architect/vuls/models"
-	"gopkg.in/gomail.v2"
 )
 
 // MailWriter send mail
@@ -33,37 +34,53 @@ type MailWriter struct{}
 func (w MailWriter) Write(scanResults []models.ScanResult) (err error) {
 	conf := config.Conf
 	for _, s := range scanResults {
-		m := gomail.NewMessage()
-		m.SetHeader("From", conf.Mail.From)
-		m.SetHeader("To", conf.Mail.To...)
-		m.SetHeader("Cc", conf.Mail.Cc...)
+		to := strings.Join(conf.Mail.To[:], ", ")
+		cc := strings.Join(conf.Mail.Cc[:], ", ")
+		mailAddresses := append(conf.Mail.To, conf.Mail.Cc...)
+		if _, err := mail.ParseAddressList(strings.Join(mailAddresses[:], ", ")); err != nil {
+			return fmt.Errorf("Failed to parse email addresses: %s", err)
+		}
 
 		subject := fmt.Sprintf("%s%s %s",
 			conf.Mail.SubjectPrefix,
-			s.ServerName,
+			s.ServerInfo(),
 			s.CveSummary(),
 		)
-		m.SetHeader("Subject", subject)
+
+		headers := make(map[string]string)
+		headers["From"] = conf.Mail.From
+		headers["To"] = to
+		headers["Cc"] = cc
+		headers["Subject"] = subject
+
+		var message string
+		for k, v := range headers {
+			message += fmt.Sprintf("%s: %s\r\n", k, v)
+		}
 
 		var body string
 		if body, err = toPlainText(s); err != nil {
 			return err
 		}
-		m.SetBody("text/plain", body)
-		port, _ := strconv.Atoi(conf.Mail.SMTPPort)
-		d := gomail.NewPlainDialer(
-			conf.Mail.SMTPAddr,
-			port,
-			conf.Mail.User,
-			conf.Mail.Password,
+		message += "\r\n" + body
+
+		smtpServer := net.JoinHostPort(conf.Mail.SMTPAddr, conf.Mail.SMTPPort)
+
+		err := smtp.SendMail(
+			smtpServer,
+			smtp.PlainAuth(
+				"",
+				conf.Mail.User,
+				conf.Mail.Password,
+				conf.Mail.SMTPAddr,
+			),
+			conf.Mail.From,
+			conf.Mail.To,
+			[]byte(message),
 		)
 
-		d.TLSConfig = &tls.Config{
-			InsecureSkipVerify: true,
-		}
-
-		if err := d.DialAndSend(m); err != nil {
-			panic(err)
+		if err != nil {
+			return fmt.Errorf("Failed to send emails: %s", err)
 		}
 	}
 	return nil

report/s3.go

@@ -0,0 +1,102 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package report
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"time"
+
+	"github.com/aws/aws-sdk-go/aws"
+	"github.com/aws/aws-sdk-go/aws/credentials"
+	"github.com/aws/aws-sdk-go/aws/session"
+	"github.com/aws/aws-sdk-go/service/s3"
+
+	c "github.com/future-architect/vuls/config"
+	"github.com/future-architect/vuls/models"
+)
+
+// CheckIfBucketExists check the existence of S3 bucket
+func CheckIfBucketExists() error {
+	svc := getS3()
+	result, err := svc.ListBuckets(&s3.ListBucketsInput{})
+	if err != nil {
+		return fmt.Errorf(
+			"Failed to list buckets. err: %s, profile: %s, region: %s",
+			err, c.Conf.AwsProfile, c.Conf.AwsRegion)
+	}
+
+	found := false
+	for _, bucket := range result.Buckets {
+		if *bucket.Name == c.Conf.S3Bucket {
+			found = true
+			break
+		}
+	}
+	if !found {
+		return fmt.Errorf(
+			"Failed to find the buckets. profile: %s, region: %s, bukdet: %s",
+			c.Conf.AwsProfile, c.Conf.AwsRegion, c.Conf.S3Bucket)
+	}
+	return nil
+}
+
+// S3Writer writes results to S3
+type S3Writer struct{}
+
+func getS3() *s3.S3 {
+	return s3.New(session.New(&aws.Config{
+		Region:      aws.String(c.Conf.AwsRegion),
+		Credentials: credentials.NewSharedCredentials("", c.Conf.AwsProfile),
+	}))
+}
+
+// Write results to S3
+func (w S3Writer) Write(scanResults []models.ScanResult) (err error) {
+
+	var jsonBytes []byte
+	if jsonBytes, err = json.Marshal(scanResults); err != nil {
+		return fmt.Errorf("Failed to Marshal to JSON: %s", err)
+	}
+
+	// http://docs.aws.amazon.com/sdk-for-go/latest/v1/developerguide/common-examples.title.html
+	svc := getS3()
+	timestr := time.Now().Format("20060102_1504")
+	for _, r := range scanResults {
+		key := ""
+		if len(r.Container.ContainerID) == 0 {
+			key = fmt.Sprintf("%s/%s.json", timestr, r.ServerName)
+		} else {
+			key = fmt.Sprintf("%s/%s_%s.json", timestr, r.ServerName, r.Container.Name)
+		}
+
+		if jsonBytes, err = json.Marshal(r); err != nil {
+			return fmt.Errorf("Failed to Marshal to JSON: %s", err)
+		}
+		_, err = svc.PutObject(&s3.PutObjectInput{
+			Bucket: &c.Conf.S3Bucket,
+			Key:    &key,
+			Body:   bytes.NewReader(jsonBytes),
+		})
+		if err != nil {
+			return fmt.Errorf("Failed to upload data to %s/%s, %s", c.Conf.S3Bucket, key, err)
+		}
+	}
+	return nil
+}

report/slack.go

@@ -59,7 +59,6 @@ type SlackWriter struct{}
 func (w SlackWriter) Write(scanResults []models.ScanResult) error {
 	conf := config.Conf.Slack
 	for _, s := range scanResults {
-
 		channel := conf.Channel
 		if channel == "${servername}" {
 			channel = fmt.Sprintf("#%s", s.ServerName)
@@ -80,7 +79,7 @@ func (w SlackWriter) Write(scanResults []models.ScanResult) error {
 				Send(string(jsonBody)).End()
 			if resp.StatusCode != 200 {
 				log.Errorf("Resonse body: %s", body)
-				if len(errs) > 0 {
+				if 0 < len(errs) {
 					return errs[0]
 				}
 			}
@@ -97,25 +96,22 @@ func (w SlackWriter) Write(scanResults []models.ScanResult) error {
 }
 
 func msgText(r models.ScanResult) string {
-
 	notifyUsers := ""
 	if 0 < len(r.KnownCves) || 0 < len(r.UnknownCves) {
 		notifyUsers = getNotifyUsers(config.Conf.Slack.NotifyUsers)
 	}
 
-	hostinfo := fmt.Sprintf(
-		"*%s* (%s %s)",
-		r.ServerName,
-		r.Family,
-		r.Release,
-	)
-	return fmt.Sprintf("%s\n%s\n>%s", notifyUsers, hostinfo, r.CveSummary())
+	serverInfo := fmt.Sprintf("*%s*", r.ServerInfo())
+	return fmt.Sprintf("%s\n%s\n>%s", notifyUsers, serverInfo, r.CveSummary())
 }
 
 func toSlackAttachments(scanResult models.ScanResult) (attaches []*attachment) {
+	cves := scanResult.KnownCves
+	if !config.Conf.IgnoreUnscoredCves {
+		cves = append(cves, scanResult.UnknownCves...)
+	}
 
-	scanResult.KnownCves = append(scanResult.KnownCves, scanResult.UnknownCves...)
-	for _, cveInfo := range scanResult.KnownCves {
+	for _, cveInfo := range cves {
 		cveID := cveInfo.CveDetail.CveID
 
 		curentPackages := []string{}
@@ -176,16 +172,15 @@ func attachmentText(cveInfo models.CveInfo, osFamily string) string {
 
 	switch {
 	case config.Conf.Lang == "ja" &&
-		cveInfo.CveDetail.Jvn.ID != 0 &&
-		0 < cveInfo.CveDetail.CvssScore("ja"):
+		0 < cveInfo.CveDetail.Jvn.CvssScore():
 
 		jvn := cveInfo.CveDetail.Jvn
 		return fmt.Sprintf("*%4.1f (%s)* <%s|%s>\n%s\n%s",
 			cveInfo.CveDetail.CvssScore(config.Conf.Lang),
-			jvn.Severity,
-			fmt.Sprintf(cvssV2CalcURLTemplate, cveInfo.CveDetail.CveID, jvn.Vector),
-			jvn.Vector,
-			jvn.Title,
+			jvn.CvssSeverity(),
+			fmt.Sprintf(cvssV2CalcURLTemplate, cveInfo.CveDetail.CveID, jvn.CvssVector()),
+			jvn.CvssVector(),
+			jvn.CveTitle(),
 			linkText,
 		)
 
@@ -193,20 +188,31 @@ func attachmentText(cveInfo models.CveInfo, osFamily string) string {
 		nvd := cveInfo.CveDetail.Nvd
 		return fmt.Sprintf("*%4.1f (%s)* <%s|%s>\n%s\n%s",
 			cveInfo.CveDetail.CvssScore(config.Conf.Lang),
-			nvd.Severity(),
+			nvd.CvssSeverity(),
 			fmt.Sprintf(cvssV2CalcURLTemplate, cveInfo.CveDetail.CveID, nvd.CvssVector()),
 			nvd.CvssVector(),
-			nvd.Summary,
+			nvd.CveSummary(),
 			linkText,
 		)
 	default:
 		nvd := cveInfo.CveDetail.Nvd
-		return fmt.Sprintf("?\n%s\n%s", nvd.Summary, linkText)
+		return fmt.Sprintf("?\n%s\n%s", nvd.CveSummary(), linkText)
 	}
 }
 
 func links(cveInfo models.CveInfo, osFamily string) string {
 	links := []string{}
+
+	cweID := cveInfo.CveDetail.CweID()
+	if 0 < len(cweID) {
+		links = append(links, fmt.Sprintf("<%s|%s>",
+			cweURL(cweID), cweID))
+		if config.Conf.Lang == "ja" {
+			links = append(links, fmt.Sprintf("<%s|%s(JVN)>",
+				cweJvnURL(cweID), cweID))
+		}
+	}
+
 	cveID := cveInfo.CveDetail.CveID
 	if config.Conf.Lang == "ja" && 0 < len(cveInfo.CveDetail.Jvn.Link()) {
 		jvn := fmt.Sprintf("<%s|JVN>", cveInfo.CveDetail.Jvn.Link())

report/stdout.go

@@ -23,10 +23,10 @@ import (
 	"github.com/future-architect/vuls/models"
 )
 
-// TextWriter write to stdout
-type TextWriter struct{}
+// StdoutWriter write to stdout
+type StdoutWriter struct{}
 
-func (w TextWriter) Write(scanResults []models.ScanResult) error {
+func (w StdoutWriter) Write(scanResults []models.ScanResult) error {
 	for _, s := range scanResults {
 		text, err := toPlainText(s)
 		if err != nil {

report/textfile.go

@@ -0,0 +1,64 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package report
+
+import (
+	"fmt"
+	"io/ioutil"
+	"path/filepath"
+	"strings"
+	"time"
+
+	"github.com/future-architect/vuls/models"
+)
+
+// TextFileWriter writes results to file.
+type TextFileWriter struct {
+	ScannedAt time.Time
+}
+
+func (w TextFileWriter) Write(scanResults []models.ScanResult) (err error) {
+	path, err := ensureResultDir(w.ScannedAt)
+	all := []string{}
+	for _, r := range scanResults {
+		textFilePath := ""
+		if len(r.Container.ContainerID) == 0 {
+			textFilePath = filepath.Join(path, fmt.Sprintf("%s.txt", r.ServerName))
+		} else {
+			textFilePath = filepath.Join(path,
+				fmt.Sprintf("%s_%s.txt", r.ServerName, r.Container.Name))
+		}
+		text, err := toPlainText(r)
+		if err != nil {
+			return err
+		}
+		all = append(all, text)
+		b := []byte(text)
+		if err := ioutil.WriteFile(textFilePath, b, 0600); err != nil {
+			return fmt.Errorf("Failed to write text files. path: %s, err: %s", textFilePath, err)
+		}
+	}
+
+	text := strings.Join(all, "\n\n")
+	b := []byte(text)
+	allPath := filepath.Join(path, "all.txt")
+	if err := ioutil.WriteFile(allPath, b, 0600); err != nil {
+		return fmt.Errorf("Failed to write text files. path: %s, err: %s", allPath, err)
+	}
+	return nil
+}

report/tui.go

@@ -20,13 +20,13 @@ package report
 import (
 	"bytes"
 	"fmt"
+	"path/filepath"
 	"strings"
 	"text/template"
 	"time"
 
 	log "github.com/Sirupsen/logrus"
 	"github.com/future-architect/vuls/config"
-	"github.com/future-architect/vuls/db"
 	"github.com/future-architect/vuls/models"
 	"github.com/google/subcommands"
 	"github.com/gosuri/uitable"
@@ -40,42 +40,55 @@ var currentCveInfo int
 var currentDetailLimitY int
 
 // RunTui execute main logic
-func RunTui() subcommands.ExitStatus {
+func RunTui(jsonDirName string) subcommands.ExitStatus {
 	var err error
-	scanHistory, err = latestScanHistory()
+	scanHistory, err = selectScanHistory(jsonDirName)
 	if err != nil {
-		log.Fatal(err)
+		log.Errorf("%s", err)
 		return subcommands.ExitFailure
 	}
 
-	g := gocui.NewGui()
-	if err := g.Init(); err != nil {
-		log.Panicln(err)
+	g, err := gocui.NewGui()
+	if err != nil {
+		log.Errorf("%s", err)
+		return subcommands.ExitFailure
 	}
 	defer g.Close()
 
-	g.SetLayout(layout)
+	g.SetManagerFunc(layout)
 	if err := keybindings(g); err != nil {
-		log.Panicln(err)
+		log.Errorf("%s", err)
+		return subcommands.ExitFailure
 	}
 	g.SelBgColor = gocui.ColorGreen
 	g.SelFgColor = gocui.ColorBlack
 	g.Cursor = true
 
 	if err := g.MainLoop(); err != nil && err != gocui.ErrQuit {
-		log.Panicln(err)
+		log.Errorf("%s", err)
 		return subcommands.ExitFailure
 	}
 
 	return subcommands.ExitSuccess
 }
 
-func latestScanHistory() (latest models.ScanHistory, err error) {
-	if err := db.OpenDB(); err != nil {
-		return latest, fmt.Errorf(
-			"Failed to open DB. datafile: %s, err: %s", config.Conf.DBPath, err)
+func selectScanHistory(jsonDirName string) (latest models.ScanHistory, err error) {
+	var jsonDir string
+	if 0 < len(jsonDirName) {
+		jsonDir = filepath.Join(config.Conf.ResultsDir, jsonDirName)
+	} else {
+		var jsonDirs JSONDirs
+		if jsonDirs, err = GetValidJSONDirs(); err != nil {
+			return
+		}
+		if len(jsonDirs) == 0 {
+			return latest, fmt.Errorf("No scan results are found in %s", config.Conf.ResultsDir)
+		}
+		jsonDir = jsonDirs[0]
+	}
+	if latest, err = LoadOneScanHistory(jsonDir); err != nil {
+		return
 	}
-	latest, err = db.SelectLatestScanHistory()
 	return
 }
 
@@ -163,35 +176,41 @@ func keybindings(g *gocui.Gui) (err error) {
 }
 
 func nextView(g *gocui.Gui, v *gocui.View) error {
+	var err error
+
 	if v == nil {
-		return g.SetCurrentView("side")
+		_, err = g.SetCurrentView("side")
 	}
 	switch v.Name() {
 	case "side":
-		return g.SetCurrentView("summary")
+		_, err = g.SetCurrentView("summary")
 	case "summary":
-		return g.SetCurrentView("detail")
+		_, err = g.SetCurrentView("detail")
 	case "detail":
-		return g.SetCurrentView("side")
+		_, err = g.SetCurrentView("side")
 	default:
-		return g.SetCurrentView("summary")
+		_, err = g.SetCurrentView("summary")
 	}
+	return err
 }
 
 func previousView(g *gocui.Gui, v *gocui.View) error {
+	var err error
+
 	if v == nil {
-		return g.SetCurrentView("side")
+		_, err = g.SetCurrentView("side")
 	}
 	switch v.Name() {
 	case "side":
-		return g.SetCurrentView("side")
+		_, err = g.SetCurrentView("side")
 	case "summary":
-		return g.SetCurrentView("side")
+		_, err = g.SetCurrentView("side")
 	case "detail":
-		return g.SetCurrentView("summary")
+		_, err = g.SetCurrentView("summary")
 	default:
-		return g.SetCurrentView("side")
+		_, err = g.SetCurrentView("side")
 	}
+	return err
 }
 
 func movable(v *gocui.View, nextY int) (ok bool, yLimit int) {
@@ -203,7 +222,7 @@ func movable(v *gocui.View, nextY int) (ok bool, yLimit int) {
 		}
 		return true, yLimit
 	case "summary":
-		yLimit = len(currentScanResult.KnownCves) - 1
+		yLimit = len(currentScanResult.AllCves()) - 1
 		if yLimit < nextY {
 			return false, yLimit
 		}
@@ -332,7 +351,7 @@ func cursorUp(g *gocui.Gui, v *gocui.View) error {
 	if v != nil {
 		ox, oy := v.Origin()
 		cx, cy := v.Cursor()
-		if err := v.SetCursor(cx, cy-1); err != nil && oy > 0 {
+		if err := v.SetCursor(cx, cy-1); err != nil && 0 < oy {
 			if err := v.SetOrigin(ox, oy-1); err != nil {
 				return err
 			}
@@ -360,7 +379,7 @@ func cursorPageUp(g *gocui.Gui, v *gocui.View) error {
 func previousSummary(g *gocui.Gui, v *gocui.View) error {
 	if v != nil {
 		// cursor to summary
-		if err := g.SetCurrentView("summary"); err != nil {
+		if _, err := g.SetCurrentView("summary"); err != nil {
 			return err
 		}
 		// move next line
@@ -368,7 +387,7 @@ func previousSummary(g *gocui.Gui, v *gocui.View) error {
 			return err
 		}
 		// cursor to detail
-		if err := g.SetCurrentView("detail"); err != nil {
+		if _, err := g.SetCurrentView("detail"); err != nil {
 			return err
 		}
 	}
@@ -378,7 +397,7 @@ func previousSummary(g *gocui.Gui, v *gocui.View) error {
 func nextSummary(g *gocui.Gui, v *gocui.View) error {
 	if v != nil {
 		// cursor to summary
-		if err := g.SetCurrentView("summary"); err != nil {
+		if _, err := g.SetCurrentView("summary"); err != nil {
 			return err
 		}
 		// move next line
@@ -386,7 +405,7 @@ func nextSummary(g *gocui.Gui, v *gocui.View) error {
 			return err
 		}
 		// cursor to detail
-		if err := g.SetCurrentView("detail"); err != nil {
+		if _, err := g.SetCurrentView("detail"); err != nil {
 			return err
 		}
 	}
@@ -410,7 +429,7 @@ func changeHost(g *gocui.Gui, v *gocui.View) error {
 	serverName := strings.TrimSpace(l)
 
 	for _, r := range scanHistory.ScanResults {
-		if serverName == r.ServerName {
+		if serverName == strings.TrimSpace(r.ServerInfoTui()) {
 			currentScanResult = r
 			break
 		}
@@ -451,7 +470,7 @@ func getLine(g *gocui.Gui, v *gocui.View) error {
 			return err
 		}
 		fmt.Fprintln(v, l)
-		if err := g.SetCurrentView("msg"); err != nil {
+		if _, err := g.SetCurrentView("msg"); err != nil {
 			return err
 		}
 	}
@@ -473,7 +492,7 @@ func showMsg(g *gocui.Gui, v *gocui.View) error {
 			return err
 		}
 		fmt.Fprintln(v, l)
-		if err := g.SetCurrentView("msg"); err != nil {
+		if _, err := g.SetCurrentView("msg"); err != nil {
 			return err
 		}
 	}
@@ -484,7 +503,7 @@ func delMsg(g *gocui.Gui, v *gocui.View) error {
 	if err := g.DeleteView("msg"); err != nil {
 		return err
 	}
-	if err := g.SetCurrentView("summary"); err != nil {
+	if _, err := g.SetCurrentView("summary"); err != nil {
 		return err
 	}
 	return nil
@@ -509,17 +528,17 @@ func layout(g *gocui.Gui) error {
 
 func setSideLayout(g *gocui.Gui) error {
 	_, maxY := g.Size()
-	if v, err := g.SetView("side", -1, -1, 30, maxY); err != nil {
+	if v, err := g.SetView("side", -1, -1, 40, maxY); err != nil {
 		if err != gocui.ErrUnknownView {
 			return err
 		}
 		v.Highlight = true
 
 		for _, result := range scanHistory.ScanResults {
-			fmt.Fprintln(v, result.ServerName)
+			fmt.Fprintln(v, result.ServerInfoTui())
 		}
 		currentScanResult = scanHistory.ScanResults[0]
-		if err := g.SetCurrentView("side"); err != nil {
+		if _, err := g.SetCurrentView("side"); err != nil {
 			return err
 		}
 	}
@@ -528,12 +547,12 @@ func setSideLayout(g *gocui.Gui) error {
 
 func setSummaryLayout(g *gocui.Gui) error {
 	maxX, maxY := g.Size()
-	if v, err := g.SetView("summary", 30, -1, maxX, int(float64(maxY)*0.2)); err != nil {
+	if v, err := g.SetView("summary", 40, -1, maxX, int(float64(maxY)*0.2)); err != nil {
 		if err != gocui.ErrUnknownView {
 			return err
 		}
 
-		lines := summaryLines(currentScanResult)
+		lines := summaryLines()
 		fmt.Fprintf(v, lines)
 
 		v.Highlight = true
@@ -543,40 +562,40 @@ func setSummaryLayout(g *gocui.Gui) error {
 	return nil
 }
 
-func summaryLines(data models.ScanResult) string {
+func summaryLines() string {
 	stable := uitable.New()
 	stable.MaxColWidth = 1000
 	stable.Wrap = false
 
 	indexFormat := ""
-	if len(data.KnownCves) < 10 {
+	if len(currentScanResult.AllCves()) < 10 {
 		indexFormat = "[%1d]"
-	} else if len(data.KnownCves) < 100 {
+	} else if len(currentScanResult.AllCves()) < 100 {
 		indexFormat = "[%2d]"
 	} else {
 		indexFormat = "[%3d]"
 	}
 
-	for i, d := range data.KnownCves {
+	for i, d := range currentScanResult.AllCves() {
 		var cols []string
 		//  packs := []string{}
 		//  for _, pack := range d.Packages {
 		//      packs = append(packs, pack.Name)
 		//  }
 		if config.Conf.Lang == "ja" && 0 < d.CveDetail.Jvn.CvssScore() {
-			summary := d.CveDetail.Jvn.Title
+			summary := d.CveDetail.Jvn.CveTitle()
 			cols = []string{
 				fmt.Sprintf(indexFormat, i+1),
 				d.CveDetail.CveID,
 				fmt.Sprintf("|  %-4.1f(%s)",
 					d.CveDetail.CvssScore(config.Conf.Lang),
-					d.CveDetail.Jvn.Severity,
+					d.CveDetail.Jvn.CvssSeverity(),
 				),
 				//  strings.Join(packs, ","),
 				summary,
 			}
 		} else {
-			summary := d.CveDetail.Nvd.Summary
+			summary := d.CveDetail.Nvd.CveSummary()
 
 			var cvssScore string
 			if d.CveDetail.CvssScore("en") <= 0 {
@@ -584,7 +603,7 @@ func summaryLines(data models.ScanResult) string {
 			} else {
 				cvssScore = fmt.Sprintf("| %-4.1f(%s)",
 					d.CveDetail.CvssScore(config.Conf.Lang),
-					d.CveDetail.Nvd.Severity(),
+					d.CveDetail.Nvd.CvssSeverity(),
 				)
 			}
 
@@ -616,7 +635,7 @@ func setDetailLayout(g *gocui.Gui) error {
 	_, oy := summaryView.Origin()
 	currentCveInfo = cy + oy
 
-	if v, err := g.SetView("detail", 30, int(float64(maxY)*0.2), maxX, maxY); err != nil {
+	if v, err := g.SetView("detail", 40, int(float64(maxY)*0.2), maxX, maxY); err != nil {
 		if err != gocui.ErrUnknownView {
 			return err
 		}
@@ -643,6 +662,7 @@ type dataForTmpl struct {
 	CvssVector       string
 	CvssSeverity     string
 	Summary          string
+	CweURL           string
 	VulnSiteLinks    []string
 	References       []cve.Reference
 	Packages         []string
@@ -652,11 +672,11 @@ type dataForTmpl struct {
 }
 
 func detailLines() (string, error) {
-	if len(currentScanResult.KnownCves) == 0 {
+	if len(currentScanResult.AllCves()) == 0 {
 		return "No vulnerable packages", nil
 	}
 
-	cveInfo := currentScanResult.KnownCves[currentCveInfo]
+	cveInfo := currentScanResult.AllCves()[currentCveInfo]
 	cveID := cveInfo.CveDetail.CveID
 
 	tmpl, err := template.New("detail").Parse(detailTemplate())
@@ -670,18 +690,20 @@ func detailLines() (string, error) {
 	case config.Conf.Lang == "ja" &&
 		0 < cveInfo.CveDetail.Jvn.CvssScore():
 		jvn := cveInfo.CveDetail.Jvn
-		cvssSeverity = jvn.Severity
-		cvssVector = jvn.Vector
-		summary = fmt.Sprintf("%s\n%s", jvn.Title, jvn.Summary)
-		refs = jvn.References
+		cvssSeverity = jvn.CvssSeverity()
+		cvssVector = jvn.CvssVector()
+		summary = fmt.Sprintf("%s\n%s", jvn.CveTitle(), jvn.CveSummary())
+		refs = jvn.VulnSiteReferences()
 	default:
 		nvd := cveInfo.CveDetail.Nvd
-		cvssSeverity = nvd.Severity()
+		cvssSeverity = nvd.CvssSeverity()
 		cvssVector = nvd.CvssVector()
-		summary = nvd.Summary
-		refs = nvd.References
+		summary = nvd.CveSummary()
+		refs = nvd.VulnSiteReferences()
 	}
 
+	cweURL := cweURL(cveInfo.CveDetail.CweID())
+
 	links := []string{
 		fmt.Sprintf("[NVD]( %s )", fmt.Sprintf("%s?vulnId=%s", nvdBaseURL, cveID)),
 		fmt.Sprintf("[MITRE]( %s )", fmt.Sprintf("%s%s", mitreBaseURL, cveID)),
@@ -715,6 +737,7 @@ func detailLines() (string, error) {
 		CvssSeverity:  cvssSeverity,
 		CvssVector:    cvssVector,
 		Summary:       summary,
+		CweURL:        cweURL,
 		VulnSiteLinks: links,
 		References:    refs,
 		Packages:      packages,
@@ -746,6 +769,11 @@ Summary
 
  {{.Summary }}
 
+CWE
+--------------
+
+ {{.CweURL }}
+
 Package/CPE
 --------------
 

report/util.go

@@ -20,26 +20,54 @@ package report
 import (
 	"bytes"
 	"fmt"
+	"os"
+	"path/filepath"
 	"strings"
+	"time"
 
 	"github.com/future-architect/vuls/config"
 	"github.com/future-architect/vuls/models"
 	"github.com/gosuri/uitable"
 )
 
+func ensureResultDir(scannedAt time.Time) (path string, err error) {
+	const timeLayout = "20060102_1504"
+	jsonDirName := scannedAt.Format(timeLayout)
+
+	resultsDir := config.Conf.ResultsDir
+	if len(resultsDir) == 0 {
+		wd, _ := os.Getwd()
+		resultsDir = filepath.Join(wd, "results")
+	}
+	jsonDir := filepath.Join(resultsDir, jsonDirName)
+
+	if err := os.MkdirAll(jsonDir, 0700); err != nil {
+		return "", fmt.Errorf("Failed to create dir: %s", err)
+	}
+
+	symlinkPath := filepath.Join(resultsDir, "current")
+	if _, err := os.Lstat(symlinkPath); err == nil {
+		if err := os.Remove(symlinkPath); err != nil {
+			return "", fmt.Errorf(
+				"Failed to remove symlink. path: %s, err: %s", symlinkPath, err)
+		}
+	}
+
+	if err := os.Symlink(jsonDir, symlinkPath); err != nil {
+		return "", fmt.Errorf(
+			"Failed to create symlink: path: %s, err: %s", symlinkPath, err)
+	}
+	return jsonDir, nil
+}
+
 func toPlainText(scanResult models.ScanResult) (string, error) {
-	hostinfo := fmt.Sprintf(
-		"%s (%s %s)",
-		scanResult.ServerName,
-		scanResult.Family,
-		scanResult.Release,
-	)
+	serverInfo := scanResult.ServerInfo()
 
 	var buffer bytes.Buffer
-	for i := 0; i < len(hostinfo); i++ {
+	for i := 0; i < len(serverInfo); i++ {
 		buffer.WriteString("=")
 	}
-	header := fmt.Sprintf("%s\n%s", hostinfo, buffer.String())
+	header := fmt.Sprintf("%s\n%s", serverInfo, buffer.String())
 
 	if len(scanResult.KnownCves) == 0 && len(scanResult.UnknownCves) == 0 {
 		return fmt.Sprintf(`
@@ -53,7 +81,12 @@ No unsecure packages.
 	scoredReport, unscoredReport = toPlainTextDetails(scanResult, scanResult.Family)
 
 	scored := strings.Join(scoredReport, "\n\n")
-	unscored := strings.Join(unscoredReport, "\n\n")
+
+	unscored := ""
+	if !config.Conf.IgnoreUnscoredCves {
+		unscored = strings.Join(unscoredReport, "\n\n")
+	}
+
 	detail := fmt.Sprintf(`
 %s
 
@@ -72,30 +105,35 @@ func ToPlainTextSummary(r models.ScanResult) string {
 	stable := uitable.New()
 	stable.MaxColWidth = 84
 	stable.Wrap = true
-	cves := append(r.KnownCves, r.UnknownCves...)
+
+	cves := r.KnownCves
+	if !config.Conf.IgnoreUnscoredCves {
+		cves = append(cves, r.UnknownCves...)
+	}
+
 	for _, d := range cves {
 		var scols []string
 
 		switch {
 		case config.Conf.Lang == "ja" &&
-			d.CveDetail.Jvn.ID != 0 &&
-			0 < d.CveDetail.CvssScore("ja"):
+			0 < d.CveDetail.Jvn.CvssScore():
 
-			summary := d.CveDetail.Jvn.Title
+			summary := d.CveDetail.Jvn.CveTitle()
 			scols = []string{
 				d.CveDetail.CveID,
 				fmt.Sprintf("%-4.1f (%s)",
 					d.CveDetail.CvssScore(config.Conf.Lang),
-					d.CveDetail.Jvn.Severity,
+					d.CveDetail.Jvn.CvssSeverity(),
 				),
 				summary,
 			}
 		case 0 < d.CveDetail.CvssScore("en"):
-			summary := d.CveDetail.Nvd.Summary
+			summary := d.CveDetail.Nvd.CveSummary()
 			scols = []string{
 				d.CveDetail.CveID,
-				fmt.Sprintf("%-4.1f",
+				fmt.Sprintf("%-4.1f (%s)",
 					d.CveDetail.CvssScore(config.Conf.Lang),
+					d.CveDetail.Nvd.CvssSeverity(),
 				),
 				summary,
 			}
@@ -103,7 +141,7 @@ func ToPlainTextSummary(r models.ScanResult) string {
 			scols = []string{
 				d.CveDetail.CveID,
 				"?",
-				d.CveDetail.Nvd.Summary,
+				d.CveDetail.Nvd.CveSummary(),
 			}
 		}
 
@@ -116,12 +154,11 @@ func ToPlainTextSummary(r models.ScanResult) string {
 	return fmt.Sprintf("%s", stable)
 }
 
-//TODO Distro Advisory
 func toPlainTextDetails(data models.ScanResult, osFamily string) (scoredReport, unscoredReport []string) {
 	for _, cve := range data.KnownCves {
 		switch config.Conf.Lang {
 		case "en":
-			if cve.CveDetail.Nvd.ID != 0 {
+			if 0 < cve.CveDetail.Nvd.CvssScore() {
 				scoredReport = append(
 					scoredReport, toPlainTextDetailsLangEn(cve, osFamily))
 			} else {
@@ -129,10 +166,10 @@ func toPlainTextDetails(data models.ScanResult, osFamily string) (scoredReport,
 					scoredReport, toPlainTextUnknownCve(cve, osFamily))
 			}
 		case "ja":
-			if cve.CveDetail.Jvn.ID != 0 {
+			if 0 < cve.CveDetail.Jvn.CvssScore() {
 				scoredReport = append(
 					scoredReport, toPlainTextDetailsLangJa(cve, osFamily))
-			} else if cve.CveDetail.Nvd.ID != 0 {
+			} else if 0 < cve.CveDetail.Nvd.CvssScore() {
 				scoredReport = append(
 					scoredReport, toPlainTextDetailsLangEn(cve, osFamily))
 			} else {
@@ -170,13 +207,11 @@ func toPlainTextUnknownCve(cveInfo models.CveInfo, osFamily string) string {
 }
 
 func toPlainTextDetailsLangJa(cveInfo models.CveInfo, osFamily string) string {
-
 	cveDetail := cveInfo.CveDetail
 	cveID := cveDetail.CveID
 	jvn := cveDetail.Jvn
 
 	dtable := uitable.New()
-	//TODO resize
 	dtable.MaxColWidth = 100
 	dtable.Wrap = true
 	dtable.AddRow(cveID)
@@ -185,14 +220,16 @@ func toPlainTextDetailsLangJa(cveInfo models.CveInfo, osFamily string) string {
 		dtable.AddRow("Score",
 			fmt.Sprintf("%4.1f (%s)",
 				cveDetail.Jvn.CvssScore(),
-				jvn.Severity,
+				jvn.CvssSeverity(),
 			))
 	} else {
 		dtable.AddRow("Score", "?")
 	}
-	dtable.AddRow("Vector", jvn.Vector)
-	dtable.AddRow("Title", jvn.Title)
-	dtable.AddRow("Description", jvn.Summary)
+	dtable.AddRow("Vector", jvn.CvssVector())
+	dtable.AddRow("Title", jvn.CveTitle())
+	dtable.AddRow("Description", jvn.CveSummary())
+	dtable.AddRow(cveDetail.CweID(), cweURL(cveDetail.CweID()))
+	dtable.AddRow(cveDetail.CweID()+"(JVN)", cweJvnURL(cveDetail.CweID()))
 
 	dtable.AddRow("JVN", jvn.Link())
 	dtable.AddRow("NVD", fmt.Sprintf("%s?vulnId=%s", nvdBaseURL, cveID))
@@ -217,7 +254,6 @@ func toPlainTextDetailsLangEn(d models.CveInfo, osFamily string) string {
 	nvd := cveDetail.Nvd
 
 	dtable := uitable.New()
-	//TODO resize
 	dtable.MaxColWidth = 100
 	dtable.Wrap = true
 	dtable.AddRow(cveID)
@@ -227,14 +263,16 @@ func toPlainTextDetailsLangEn(d models.CveInfo, osFamily string) string {
 		dtable.AddRow("Score",
 			fmt.Sprintf("%4.1f (%s)",
 				cveDetail.Nvd.CvssScore(),
-				nvd.Severity(),
+				nvd.CvssSeverity(),
 			))
 	} else {
 		dtable.AddRow("Score", "?")
 	}
 
 	dtable.AddRow("Vector", nvd.CvssVector())
-	dtable.AddRow("Summary", nvd.Summary)
+	dtable.AddRow("Summary", nvd.CveSummary())
+	dtable.AddRow("CWE", cweURL(cveDetail.CweID()))
+
 	dtable.AddRow("NVD", fmt.Sprintf("%s?vulnId=%s", nvdBaseURL, cveID))
 	dtable.AddRow("MITRE", fmt.Sprintf("%s%s", mitreBaseURL, cveID))
 	dtable.AddRow("CVE Details", fmt.Sprintf("%s/%s", cveDetailsBaseURL, cveID))
@@ -306,6 +344,15 @@ func distroLinks(cveInfo models.CveInfo, osFamily string) []distroLink {
 			},
 			//  TODO Debian dsa
 		}
+	case "FreeBSD":
+		links := []distroLink{}
+		for _, advisory := range cveInfo.DistroAdvisories {
+			links = append(links, distroLink{
+				"FreeBSD-VuXML",
+				fmt.Sprintf(freeBSDVuXMLBaseURL, advisory.AdvisoryID),
+			})
+		}
+		return links
 	default:
 		return []distroLink{}
 	}
@@ -332,3 +379,12 @@ func addCpeNames(table *uitable.Table, names []models.CpeName) *uitable.Table {
 	}
 	return table
 }
+
+func cweURL(cweID string) string {
+	return fmt.Sprintf("https://cwe.mitre.org/data/definitions/%s.html",
+		strings.TrimPrefix(cweID, "CWE-"))
+}
+
+func cweJvnURL(cweID string) string {
+	return fmt.Sprintf("http://jvndb.jvn.jp/ja/cwe/%s.html", cweID)
+}

report/writer.go

@@ -31,6 +31,8 @@ const (
 
 	ubuntuSecurityBaseURL = "http://people.ubuntu.com/~ubuntu-security/cve"
 	debianTrackerBaseURL  = "https://security-tracker.debian.org/tracker"
+
+	freeBSDVuXMLBaseURL = "https://vuxml.freebsd.org/freebsd/%s.html"
 )
 
 // ResultWriter Interface

report/xml.go

@@ -0,0 +1,54 @@
+package report
+
+import (
+	"bytes"
+	"encoding/xml"
+	"fmt"
+	"io/ioutil"
+	"path/filepath"
+	"time"
+
+	"github.com/future-architect/vuls/models"
+)
+
+const (
+	vulsOpenTag  = "<vulsreport>"
+	vulsCloseTag = "</vulsreport>"
+)
+
+// XMLWriter writes results to file.
+type XMLWriter struct {
+	ScannedAt time.Time
+}
+
+func (w XMLWriter) Write(scanResults []models.ScanResult) (err error) {
+	var path string
+	if path, err = ensureResultDir(w.ScannedAt); err != nil {
+		return fmt.Errorf("Failed to make direcotory/symlink : %s", err)
+	}
+
+	for _, scanResult := range scanResults {
+		scanResult.ScannedAt = w.ScannedAt
+	}
+
+	var xmlBytes []byte
+	for _, r := range scanResults {
+		xmlPath := ""
+		if len(r.Container.ContainerID) == 0 {
+			xmlPath = filepath.Join(path, fmt.Sprintf("%s.xml", r.ServerName))
+		} else {
+			xmlPath = filepath.Join(path,
+				fmt.Sprintf("%s_%s.xml", r.ServerName, r.Container.Name))
+		}
+
+		if xmlBytes, err = xml.Marshal(r); err != nil {
+			return fmt.Errorf("Failed to Marshal to XML: %s", err)
+		}
+
+		allBytes := bytes.Join([][]byte{[]byte(xml.Header + vulsOpenTag), xmlBytes, []byte(vulsCloseTag)}, []byte{})
+		if err := ioutil.WriteFile(xmlPath, allBytes, 0600); err != nil {
+			return fmt.Errorf("Failed to write XML. path: %s, err: %s", xmlPath, err)
+		}
+	}
+	return nil
+}

scan/base.go

@@ -0,0 +1,340 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package scan
+
+import (
+	"fmt"
+	"regexp"
+	"sort"
+	"strings"
+	"time"
+
+	"github.com/Sirupsen/logrus"
+	"github.com/future-architect/vuls/config"
+	"github.com/future-architect/vuls/cveapi"
+	"github.com/future-architect/vuls/models"
+)
+
+type base struct {
+	ServerInfo config.ServerInfo
+	Distro     config.Distro
+	Platform   models.Platform
+
+	lackDependencies []string
+	osPackages
+
+	log  *logrus.Entry
+	errs []error
+}
+
+func (l *base) ssh(cmd string, sudo bool) sshResult {
+	return sshExec(l.ServerInfo, cmd, sudo, l.log)
+}
+
+func (l *base) setServerInfo(c config.ServerInfo) {
+	l.ServerInfo = c
+}
+
+func (l base) getServerInfo() config.ServerInfo {
+	return l.ServerInfo
+}
+
+func (l *base) setDistro(fam, rel string) {
+	d := config.Distro{
+		Family:  fam,
+		Release: rel,
+	}
+	l.Distro = d
+
+	s := l.getServerInfo()
+	s.Distro = d
+	l.setServerInfo(s)
+}
+
+func (l base) getDistro() config.Distro {
+	return l.Distro
+}
+
+func (l *base) setPlatform(p models.Platform) {
+	l.Platform = p
+}
+
+func (l base) getPlatform() models.Platform {
+	return l.Platform
+}
+
+func (l base) getLackDependencies() []string {
+	return l.lackDependencies
+}
+
+func (l base) allContainers() (containers []config.Container, err error) {
+	switch l.ServerInfo.Container.Type {
+	case "", "docker":
+		stdout, err := l.dockerPs("-a --format '{{.ID}} {{.Names}}'")
+		if err != nil {
+			return containers, err
+		}
+		return l.parseDockerPs(stdout)
+	default:
+		return containers, fmt.Errorf(
+			"Not supported yet: %s", l.ServerInfo.Container.Type)
+	}
+}
+
+func (l *base) runningContainers() (containers []config.Container, err error) {
+	switch l.ServerInfo.Container.Type {
+	case "", "docker":
+		stdout, err := l.dockerPs("--format '{{.ID}} {{.Names}}'")
+		if err != nil {
+			return containers, err
+		}
+		return l.parseDockerPs(stdout)
+	default:
+		return containers, fmt.Errorf(
+			"Not supported yet: %s", l.ServerInfo.Container.Type)
+	}
+}
+
+func (l *base) exitedContainers() (containers []config.Container, err error) {
+	switch l.ServerInfo.Container.Type {
+	case "", "docker":
+		stdout, err := l.dockerPs("--filter 'status=exited' --format '{{.ID}} {{.Names}}'")
+		if err != nil {
+			return containers, err
+		}
+		return l.parseDockerPs(stdout)
+	default:
+		return containers, fmt.Errorf(
+			"Not supported yet: %s", l.ServerInfo.Container.Type)
+	}
+}
+
+func (l *base) dockerPs(option string) (string, error) {
+	cmd := fmt.Sprintf("docker ps %s", option)
+	r := l.ssh(cmd, noSudo)
+	if !r.isSuccess() {
+		return "", fmt.Errorf("Failed to SSH: %s", r)
+	}
+	return r.Stdout, nil
+}
+
+func (l *base) parseDockerPs(stdout string) (containers []config.Container, err error) {
+	lines := strings.Split(stdout, "\n")
+	for _, line := range lines {
+		fields := strings.Fields(line)
+		if len(fields) == 0 {
+			break
+		}
+		if len(fields) != 2 {
+			return containers, fmt.Errorf("Unknown format: %s", line)
+		}
+		containers = append(containers, config.Container{
+			ContainerID: fields[0],
+			Name:        fields[1],
+		})
+	}
+	return
+}
+
+func (l *base) detectPlatform() error {
+	ok, instanceID, err := l.detectRunningOnAws()
+	if err != nil {
+		return err
+	}
+	if ok {
+		l.setPlatform(models.Platform{
+			Name:       "aws",
+			InstanceID: instanceID,
+		})
+		return nil
+	}
+
+	//TODO Azure, GCP...
+	l.setPlatform(models.Platform{
+		Name: "other",
+	})
+	return nil
+}
+
+func (l base) detectRunningOnAws() (ok bool, instanceID string, err error) {
+	if r := l.ssh("type curl", noSudo); r.isSuccess() {
+		cmd := "curl --max-time 1 --retry 3 --noproxy 169.254.169.254 http://169.254.169.254/latest/meta-data/instance-id"
+		r := l.ssh(cmd, noSudo)
+		if r.isSuccess() {
+			id := strings.TrimSpace(r.Stdout)
+			if !l.isAwsInstanceID(id) {
+				return false, "", nil
+			}
+			return true, id, nil
+		}
+
+		switch r.ExitStatus {
+		case 28, 7:
+			// Not running on AWS
+			//  7   Failed to connect to host.
+			// 28  operation timeout.
+			return false, "", nil
+		}
+	}
+
+	if r := l.ssh("type wget", noSudo); r.isSuccess() {
+		cmd := "wget --tries=3 --timeout=1 --no-proxy -q -O - http://169.254.169.254/latest/meta-data/instance-id"
+		r := l.ssh(cmd, noSudo)
+		if r.isSuccess() {
+			id := strings.TrimSpace(r.Stdout)
+			if !l.isAwsInstanceID(id) {
+				return false, "", nil
+			}
+			return true, id, nil
+		}
+
+		switch r.ExitStatus {
+		case 4, 8:
+			// Not running on AWS
+			// 4   Network failure
+			// 8   Server issued an error response.
+			return false, "", nil
+		}
+	}
+	return false, "", fmt.Errorf(
+		"Failed to curl or wget to AWS instance metadata on %s. container: %s",
+		l.ServerInfo.ServerName, l.ServerInfo.Container.Name)
+}
+
+// http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/resource-ids.html
+var awsInstanceIDPattern = regexp.MustCompile(`^i-[0-9a-f]+$`)
+
+func (l base) isAwsInstanceID(str string) bool {
+	return awsInstanceIDPattern.MatchString(str)
+}
+
+func (l *base) convertToModel() (models.ScanResult, error) {
+	var scoredCves, unscoredCves, ignoredCves models.CveInfos
+	for _, p := range l.UnsecurePackages {
+		// ignoreCves
+		found := false
+		for _, icve := range l.getServerInfo().IgnoreCves {
+			if icve == p.CveDetail.CveID {
+				ignoredCves = append(ignoredCves, models.CveInfo{
+					CveDetail:        p.CveDetail,
+					Packages:         p.Packs,
+					DistroAdvisories: p.DistroAdvisories,
+				})
+				found = true
+				break
+			}
+		}
+		if found {
+			continue
+		}
+
+		// unscoredCves
+		if p.CveDetail.CvssScore(config.Conf.Lang) <= 0 {
+			unscoredCves = append(unscoredCves, models.CveInfo{
+				CveDetail:        p.CveDetail,
+				Packages:         p.Packs,
+				DistroAdvisories: p.DistroAdvisories,
+			})
+			continue
+		}
+
+		cpenames := []models.CpeName{}
+		for _, cpename := range p.CpeNames {
+			cpenames = append(cpenames,
+				models.CpeName{Name: cpename})
+		}
+
+		// scoredCves
+		cve := models.CveInfo{
+			CveDetail:        p.CveDetail,
+			Packages:         p.Packs,
+			DistroAdvisories: p.DistroAdvisories,
+			CpeNames:         cpenames,
+		}
+		scoredCves = append(scoredCves, cve)
+	}
+
+	container := models.Container{
+		ContainerID: l.ServerInfo.Container.ContainerID,
+		Name:        l.ServerInfo.Container.Name,
+	}
+
+	sort.Sort(scoredCves)
+	sort.Sort(unscoredCves)
+	sort.Sort(ignoredCves)
+
+	return models.ScanResult{
+		ServerName:  l.ServerInfo.ServerName,
+		ScannedAt:   time.Now(),
+		Family:      l.Distro.Family,
+		Release:     l.Distro.Release,
+		Container:   container,
+		Platform:    l.Platform,
+		KnownCves:   scoredCves,
+		UnknownCves: unscoredCves,
+		IgnoredCves: ignoredCves,
+		Optional:    l.ServerInfo.Optional,
+	}, nil
+}
+
+// scanVulnByCpeName search vulnerabilities that specified in config file.
+func (l *base) scanVulnByCpeName() error {
+	unsecurePacks := CvePacksList{}
+
+	serverInfo := l.getServerInfo()
+	cpeNames := serverInfo.CpeNames
+
+	// remove duplicate
+	set := map[string]CvePacksInfo{}
+
+	for _, name := range cpeNames {
+		details, err := cveapi.CveClient.FetchCveDetailsByCpeName(name)
+		if err != nil {
+			return err
+		}
+		for _, detail := range details {
+			if val, ok := set[detail.CveID]; ok {
+				names := val.CpeNames
+				names = append(names, name)
+				val.CpeNames = names
+				set[detail.CveID] = val
+			} else {
+				set[detail.CveID] = CvePacksInfo{
+					CveID:     detail.CveID,
+					CveDetail: detail,
+					CpeNames:  []string{name},
+				}
+			}
+		}
+	}
+
+	for key := range set {
+		unsecurePacks = append(unsecurePacks, set[key])
+	}
+	unsecurePacks = append(unsecurePacks, l.UnsecurePackages...)
+	l.setUnsecurePackages(unsecurePacks)
+	return nil
+}
+
+func (l *base) setErrs(errs []error) {
+	l.errs = errs
+}
+
+func (l base) getErrs() []error {
+	return l.errs
+}

scan/base_test.go

@@ -0,0 +1,80 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package scan
+
+import (
+	"reflect"
+	"testing"
+
+	"github.com/future-architect/vuls/config"
+)
+
+func TestParseDockerPs(t *testing.T) {
+
+	var test = struct {
+		in       string
+		expected []config.Container
+	}{
+		`c7ca0992415a romantic_goldberg
+f570ae647edc agitated_lovelace`,
+		[]config.Container{
+			{
+				ContainerID: "c7ca0992415a",
+				Name:        "romantic_goldberg",
+			},
+			{
+				ContainerID: "f570ae647edc",
+				Name:        "agitated_lovelace",
+			},
+		},
+	}
+
+	r := newRedhat(config.ServerInfo{})
+	actual, err := r.parseDockerPs(test.in)
+	if err != nil {
+		t.Errorf("Error occurred. in: %s, err: %s", test.in, err)
+		return
+	}
+	for i, e := range test.expected {
+		if !reflect.DeepEqual(e, actual[i]) {
+			t.Errorf("expected %v, actual %v", e, actual[i])
+		}
+	}
+}
+
+func TestIsAwsInstanceID(t *testing.T) {
+	var tests = []struct {
+		in       string
+		expected bool
+	}{
+		{"i-1234567a", true},
+		{"i-1234567890abcdef0", true},
+		{"i-1234567890abcdef0000000", true},
+		{"e-1234567890abcdef0", false},
+		{"i-1234567890abcdef0 foo bar", false},
+		{"no data", false},
+	}
+
+	r := newRedhat(config.ServerInfo{})
+	for _, tt := range tests {
+		actual := r.isAwsInstanceID(tt.in)
+		if tt.expected != actual {
+			t.Errorf("expected %t, actual %t, str: %s", tt.expected, actual, tt.in)
+		}
+	}
+}

scan/debian.go

@@ -20,11 +20,11 @@ package scan
 import (
 	"fmt"
 	"regexp"
-	"sort"
 	"strconv"
 	"strings"
 	"time"
 
+	"github.com/future-architect/vuls/cache"
 	"github.com/future-architect/vuls/config"
 	"github.com/future-architect/vuls/cveapi"
 	"github.com/future-architect/vuls/models"
@@ -33,29 +33,32 @@ import (
 
 // inherit OsTypeInterface
 type debian struct {
-	linux
+	base
 }
 
 // NewDebian is constructor
 func newDebian(c config.ServerInfo) *debian {
 	d := &debian{}
 	d.log = util.NewCustomLogger(c)
+	d.setServerInfo(c)
 	return d
 }
 
 // Ubuntu, Debian
 // https://github.com/serverspec/specinfra/blob/master/lib/specinfra/helper/detect_os/debian.rb
-func detectDebian(c config.ServerInfo) (itsMe bool, deb osTypeInterface) {
-
+func detectDebian(c config.ServerInfo) (itsMe bool, deb osTypeInterface, err error) {
 	deb = newDebian(c)
 
-	// set sudo option flag
-	c.SudoOpt = config.SudoOption{ExecBySudo: true}
-	deb.setServerInfo(c)
-
 	if r := sshExec(c, "ls /etc/debian_version", noSudo); !r.isSuccess() {
-		Log.Debugf("Not Debian like Linux. Host: %s:%s", c.Host, c.Port)
-		return false, deb
+		if r.Error != nil {
+			return false, deb, r.Error
+		}
+		if r.ExitStatus == 255 {
+			return false, deb, fmt.Errorf(
+				"Unable to connect via SSH. Check SSH settings. %s", r)
+		}
+		Log.Debugf("Not Debian like Linux. %s", r)
+		return false, deb, nil
 	}
 
 	if r := sshExec(c, "lsb_release -ir", noSudo); r.isSuccess() {
@@ -63,20 +66,18 @@ func detectDebian(c config.ServerInfo) (itsMe bool, deb osTypeInterface) {
 		//  root@fa3ec524be43:/# lsb_release -ir
 		//  Distributor ID:	Ubuntu
 		//  Release:	14.04
-		re, _ := regexp.Compile(
-			`(?s)^Distributor ID:\s*(.+?)\n*Release:\s*(.+?)$`)
+		re := regexp.MustCompile(`(?s)^Distributor ID:\s*(.+?)\n*Release:\s*(.+?)$`)
 		result := re.FindStringSubmatch(trim(r.Stdout))
 
 		if len(result) == 0 {
-			deb.setDistributionInfo("debian/ubuntu", "unknown")
+			deb.setDistro("debian/ubuntu", "unknown")
 			Log.Warnf(
-				"Unknown Debian/Ubuntu version. lsb_release -ir: %s, Host: %s:%s",
-				r.Stdout, c.Host, c.Port)
+				"Unknown Debian/Ubuntu version. lsb_release -ir: %s", r)
 		} else {
 			distro := strings.ToLower(trim(result[1]))
-			deb.setDistributionInfo(distro, trim(result[2]))
+			deb.setDistro(distro, trim(result[2]))
 		}
-		return true, deb
+		return true, deb, nil
 	}
 
 	if r := sshExec(c, "cat /etc/lsb-release", noSudo); r.isSuccess() {
@@ -85,81 +86,87 @@ func detectDebian(c config.ServerInfo) (itsMe bool, deb osTypeInterface) {
 		//  DISTRIB_RELEASE=14.04
 		//  DISTRIB_CODENAME=trusty
 		//  DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"
-		re, _ := regexp.Compile(
-			`(?s)^DISTRIB_ID=(.+?)\n*DISTRIB_RELEASE=(.+?)\n.*$`)
+		re := regexp.MustCompile(`(?s)^DISTRIB_ID=(.+?)\n*DISTRIB_RELEASE=(.+?)\n.*$`)
 		result := re.FindStringSubmatch(trim(r.Stdout))
 		if len(result) == 0 {
 			Log.Warnf(
-				"Unknown Debian/Ubuntu. cat /etc/lsb-release: %s, Host: %s:%s",
-				r.Stdout, c.Host, c.Port)
-			deb.setDistributionInfo("debian/ubuntu", "unknown")
+				"Unknown Debian/Ubuntu. cat /etc/lsb-release: %s", r)
+			deb.setDistro("debian/ubuntu", "unknown")
 		} else {
 			distro := strings.ToLower(trim(result[1]))
-			deb.setDistributionInfo(distro, trim(result[2]))
+			deb.setDistro(distro, trim(result[2]))
 		}
-		return true, deb
+		return true, deb, nil
 	}
 
 	// Debian
 	cmd := "cat /etc/debian_version"
 	if r := sshExec(c, cmd, noSudo); r.isSuccess() {
-		deb.setDistributionInfo("debian", trim(r.Stdout))
-		return true, deb
+		deb.setDistro("debian", trim(r.Stdout))
+		return true, deb, nil
 	}
 
-	Log.Debugf("Not Debian like Linux. Host: %s:%s", c.Host, c.Port)
-	return false, deb
+	Log.Debugf("Not Debian like Linux: %s", c.ServerName)
+	return false, deb, nil
 }
 
 func trim(str string) string {
 	return strings.TrimSpace(str)
 }
 
+func (o *debian) checkIfSudoNoPasswd() error {
+	r := o.ssh("apt-get -v", sudo)
+	if !r.isSuccess() {
+		o.log.Errorf("sudo error on %s", r)
+		return fmt.Errorf("Failed to sudo: %s", r)
+	}
+	o.log.Infof("sudo ... OK")
+	return nil
+}
+
+func (o *debian) checkDependencies() error {
+	switch o.Distro.Family {
+	case "ubuntu":
+		return nil
+
+	case "debian":
+		// Debian needs aptitude to get changelogs.
+		// Because unable to get changelogs via apt-get changelog on Debian.
+		name := "aptitude"
+		cmd := name + " -h"
+		if r := o.ssh(cmd, noSudo); !r.isSuccess() {
+			o.lackDependencies = []string{name}
+		}
+		return nil
+
+	default:
+		return fmt.Errorf("Not implemented yet: %s", o.Distro)
+	}
+}
+
 func (o *debian) install() error {
+	if len(o.lackDependencies) == 0 {
+		return nil
+	}
 
 	// apt-get update
 	o.log.Infof("apt-get update...")
 	cmd := util.PrependProxyEnv("apt-get update")
 	if r := o.ssh(cmd, sudo); !r.isSuccess() {
-		msg := fmt.Sprintf("Failed to %s. status: %d, stdout: %s, stderr: %s",
-			cmd, r.ExitStatus, r.Stdout, r.Stderr)
+		msg := fmt.Sprintf("Failed to SSH: %s", r)
 		o.log.Errorf(msg)
 		return fmt.Errorf(msg)
 	}
 
-	if o.Family == "debian" {
-		// install aptitude
-		cmd = util.PrependProxyEnv("apt-get install --force-yes -y aptitude")
+	for _, name := range o.lackDependencies {
+		cmd = util.PrependProxyEnv("apt-get install " + name)
 		if r := o.ssh(cmd, sudo); !r.isSuccess() {
-			msg := fmt.Sprintf("Failed to %s. status: %d, stdout: %s, stderr: %s",
-				cmd, r.ExitStatus, r.Stdout, r.Stderr)
+			msg := fmt.Sprintf("Failed to SSH: %s", r)
 			o.log.Errorf(msg)
 			return fmt.Errorf(msg)
 		}
-		o.log.Infof("Installed: aptitude")
+		o.log.Infof("Installed: " + name)
 	}
-
-	// install unattended-upgrades
-	if !config.Conf.UseUnattendedUpgrades {
-		return nil
-	}
-
-	if r := o.ssh("type unattended-upgrade", noSudo); r.isSuccess() {
-		o.log.Infof(
-			"Ignored: unattended-upgrade already installed")
-		return nil
-	}
-
-	cmd = util.PrependProxyEnv(
-		"apt-get install --force-yes -y unattended-upgrades")
-	if r := o.ssh(cmd, sudo); !r.isSuccess() {
-		msg := fmt.Sprintf("Failed to %s. status: %d, stdout: %s, stderr: %s",
-			cmd, r.ExitStatus, r.Stdout, r.Stderr)
-		o.log.Errorf(msg)
-		return fmt.Errorf(msg)
-	}
-
-	o.log.Infof("Installed: unattended-upgrades")
 	return nil
 }
 
@@ -184,9 +191,7 @@ func (o *debian) scanPackages() error {
 func (o *debian) scanInstalledPackages() (packs []models.PackageInfo, err error) {
 	r := o.ssh("dpkg-query -W", noSudo)
 	if !r.isSuccess() {
-		return packs, fmt.Errorf(
-			"Failed to scan packages. status: %d, stdout:%s, stderr: %s",
-			r.ExitStatus, r.Stdout, r.Stderr)
+		return packs, fmt.Errorf("Failed to SSH: %s", r)
 	}
 
 	//  e.g.
@@ -195,7 +200,7 @@ func (o *debian) scanInstalledPackages() (packs []models.PackageInfo, err error)
 	lines := strings.Split(r.Stdout, "\n")
 	for _, line := range lines {
 		if trimmed := strings.TrimSpace(line); len(trimmed) != 0 {
-			name, version, err := o.parseScanedPackagesLine(trimmed)
+			name, version, err := o.parseScannedPackagesLine(trimmed)
 			if err != nil {
 				return nil, fmt.Errorf(
 					"Debian: Failed to parse package line: %s", line)
@@ -209,12 +214,16 @@ func (o *debian) scanInstalledPackages() (packs []models.PackageInfo, err error)
 	return
 }
 
-func (o *debian) parseScanedPackagesLine(line string) (name, version string, err error) {
-	re, _ := regexp.Compile(`^([^\t']+)\t(.+)$`)
-	result := re.FindStringSubmatch(line)
+var packageLinePattern = regexp.MustCompile(`^([^\t']+)\t(.+)$`)
+
+func (o *debian) parseScannedPackagesLine(line string) (name, version string, err error) {
+	result := packageLinePattern.FindStringSubmatch(line)
 	if len(result) == 3 {
 		// remove :amd64, i386...
-		name = regexp.MustCompile(":.+").ReplaceAllString(result[1], "")
+		name = result[1]
+		if i := strings.IndexRune(name, ':'); i >= 0 {
+			name = name[:i]
+		}
 		version = result[2]
 		return
 	}
@@ -222,52 +231,27 @@ func (o *debian) parseScanedPackagesLine(line string) (name, version string, err
 	return "", "", fmt.Errorf("Unknown format: %s", line)
 }
 
-//  unattended-upgrade command need to check security upgrades).
 func (o *debian) checkRequiredPackagesInstalled() error {
-
-	if o.Family == "debian" {
-		if r := o.ssh("test -f /usr/bin/aptitude", sudo); !r.isSuccess() {
-			msg := "aptitude is not installed"
+	if o.Distro.Family == "debian" {
+		if r := o.ssh("test -f /usr/bin/aptitude", noSudo); !r.isSuccess() {
+			msg := fmt.Sprintf("aptitude is not installed: %s", r)
 			o.log.Errorf(msg)
 			return fmt.Errorf(msg)
 		}
 	}
-
-	if !config.Conf.UseUnattendedUpgrades {
-		return nil
-	}
-
-	if r := o.ssh("type unattended-upgrade", noSudo); !r.isSuccess() {
-		msg := "unattended-upgrade is not installed"
-		o.log.Errorf(msg)
-		return fmt.Errorf(msg)
-	}
 	return nil
 }
 
-//TODO return whether already expired.
 func (o *debian) scanUnsecurePackages(packs []models.PackageInfo) ([]CvePacksInfo, error) {
-	//  cmd := prependProxyEnv(conf.HTTPProxy, "apt-get update | cat; echo 1")
+	o.log.Infof("apt-get update...")
 	cmd := util.PrependProxyEnv("apt-get update")
 	if r := o.ssh(cmd, sudo); !r.isSuccess() {
-		return nil, fmt.Errorf(
-			"Failed to %s. status: %d, stdout: %s, stderr: %s",
-			cmd, r.ExitStatus, r.Stdout, r.Stderr,
-		)
+		return nil, fmt.Errorf("Failed to SSH: %s", r)
 	}
 
-	var upgradablePackNames []string
-	var err error
-	if config.Conf.UseUnattendedUpgrades {
-		upgradablePackNames, err = o.GetUnsecurePackNamesUsingUnattendedUpgrades()
-		if err != nil {
-			return []CvePacksInfo{}, err
-		}
-	} else {
-		upgradablePackNames, err = o.GetUpgradablePackNames()
-		if err != nil {
-			return []CvePacksInfo{}, err
-		}
+	upgradablePackNames, err := o.GetUpgradablePackNames()
+	if err != nil {
+		return []CvePacksInfo{}, err
 	}
 
 	// Convert package name to PackageInfo struct
@@ -280,9 +264,18 @@ func (o *debian) scanUnsecurePackages(packs []models.PackageInfo) ([]CvePacksInf
 			}
 		}
 	}
-
 	unsecurePacks, err = o.fillCandidateVersion(unsecurePacks)
 	if err != nil {
+		return nil, fmt.Errorf("Failed to fill candidate versions. err: %s", err)
+	}
+
+	current := cache.Meta{
+		Name:   o.getServerInfo().GetServerName(),
+		Distro: o.getServerInfo().Distro,
+		Packs:  unsecurePacks,
+	}
+	o.log.Debugf("Ensure changelog cache: %s", current.Name)
+	if err := o.ensureChangelogCache(current); err != nil {
 		return nil, err
 	}
 
@@ -295,101 +288,65 @@ func (o *debian) scanUnsecurePackages(packs []models.PackageInfo) ([]CvePacksInf
 	return cvePacksInfos, nil
 }
 
-func (o *debian) fillCandidateVersion(packs []models.PackageInfo) ([]models.PackageInfo, error) {
-	reqChan := make(chan models.PackageInfo, len(packs))
-	resChan := make(chan models.PackageInfo, len(packs))
-	errChan := make(chan error, len(packs))
-	defer close(resChan)
-	defer close(errChan)
-	defer close(reqChan)
-
-	go func() {
-		for _, pack := range packs {
-			reqChan <- pack
+func (o *debian) ensureChangelogCache(current cache.Meta) error {
+	// Search from cache
+	old, found, err := cache.DB.GetMeta(current.Name)
+	if err != nil {
+		return fmt.Errorf("Failed to get meta. err: %s", err)
+	}
+	if !found {
+		o.log.Debugf("Not found in meta: %s", current.Name)
+		err = cache.DB.EnsureBuckets(current)
+		if err != nil {
+			return fmt.Errorf("Failed to ensure buckets. err: %s", err)
 		}
-	}()
-
-	timeout := time.After(5 * 60 * time.Second)
-	concurrency := 5
-	tasks := util.GenWorkers(concurrency)
-	for range packs {
-		tasks <- func() {
-			select {
-			case pack := <-reqChan:
-				func(p models.PackageInfo) {
-					cmd := fmt.Sprintf("apt-cache policy %s", p.Name)
-					r := o.ssh(cmd, sudo)
-					if !r.isSuccess() {
-						errChan <- fmt.Errorf(
-							"Failed to %s. status: %d, stdout: %s, stderr: %s",
-							cmd, r.ExitStatus, r.Stdout, r.Stderr)
-						return
-					}
-					ver, err := o.parseAptCachePolicy(r.Stdout, p.Name)
-					if err != nil {
-						errChan <- fmt.Errorf("Failed to parse %s", err)
-					}
-					p.NewVersion = ver.Candidate
-					resChan <- p
-				}(pack)
+	} else {
+		if current.Distro.Family != old.Distro.Family ||
+			current.Distro.Release != old.Distro.Release {
+			o.log.Debugf("Need to refesh meta: %s", current.Name)
+			err = cache.DB.EnsureBuckets(current)
+			if err != nil {
+				return fmt.Errorf("Failed to ensure buckets. err: %s", err)
 			}
+		} else {
+			o.log.Debugf("Reuse meta: %s", current.Name)
 		}
 	}
 
-	result := []models.PackageInfo{}
-	for i := 0; i < len(packs); i++ {
-		select {
-		case pack := <-resChan:
-			result = append(result, pack)
-			o.log.Infof("(%d/%d) Upgradable: %s-%s -> %s",
-				i+1, len(packs), pack.Name, pack.Version, pack.NewVersion)
-		case err := <-errChan:
-			return nil, err
-		case <-timeout:
-			return nil, fmt.Errorf("Timeout fillCandidateVersion")
-		}
+	if config.Conf.Debug {
+		cache.DB.PrettyPrint(current)
 	}
-	return result, nil
+	return nil
 }
 
-func (o *debian) GetUnsecurePackNamesUsingUnattendedUpgrades() (packNames []string, err error) {
-	cmd := util.PrependProxyEnv("unattended-upgrades --dry-run -d 2>&1 ")
-	release, err := strconv.ParseFloat(o.Release, 64)
-	if err != nil {
-		return packNames, fmt.Errorf(
-			"OS Release Version is invalid, %s, %s", o.Family, o.Release)
+func (o *debian) fillCandidateVersion(before models.PackageInfoList) (filled []models.PackageInfo, err error) {
+	names := []string{}
+	for _, p := range before {
+		names = append(names, p.Name)
 	}
-	switch {
-	case release < 12:
-		return packNames, fmt.Errorf(
-			"Support expired. %s, %s", o.Family, o.Release)
-
-	case 12 < release && release < 14:
-		cmd += `| grep 'pkgs that look like they should be upgraded:' |
-			sed -e 's/pkgs that look like they should be upgraded://g'`
-
-	case 14 < release:
-		cmd += `| grep 'Packages that will be upgraded:' |
-			sed -e 's/Packages that will be upgraded://g'`
-
-	default:
-		return packNames, fmt.Errorf(
-			"Not supported yet. %s, %s", o.Family, o.Release)
-	}
-
+	cmd := fmt.Sprintf("LANGUAGE=en_US.UTF-8 apt-cache policy %s", strings.Join(names, " "))
 	r := o.ssh(cmd, sudo)
-	if r.isSuccess(0, 1) {
-		packNames = strings.Split(strings.TrimSpace(r.Stdout), " ")
-		return packNames, nil
+	if !r.isSuccess() {
+		return nil, fmt.Errorf("Failed to SSH: %s.", r)
 	}
-
-	return packNames, fmt.Errorf(
-		"Failed to %s. status: %d, stdout: %s, stderr: %s",
-		cmd, r.ExitStatus, r.Stdout, r.Stderr)
+	packChangelog := o.splitAptCachePolicy(r.Stdout)
+	for k, v := range packChangelog {
+		ver, err := o.parseAptCachePolicy(v, k)
+		if err != nil {
+			return nil, fmt.Errorf("Failed to parse %s", err)
+		}
+		p, found := before.FindByName(k)
+		if !found {
+			return nil, fmt.Errorf("Not found: %s", k)
+		}
+		p.NewVersion = ver.Candidate
+		filled = append(filled, p)
+	}
+	return
 }
 
 func (o *debian) GetUpgradablePackNames() (packNames []string, err error) {
-	cmd := util.PrependProxyEnv("apt-get upgrade --dry-run")
+	cmd := util.PrependProxyEnv("LANGUAGE=en_US.UTF-8 apt-get upgrade --dry-run")
 	r := o.ssh(cmd, sudo)
 	if r.isSuccess(0, 1) {
 		return o.parseAptGetUpgrade(r.Stdout)
@@ -400,8 +357,8 @@ func (o *debian) GetUpgradablePackNames() (packNames []string, err error) {
 }
 
 func (o *debian) parseAptGetUpgrade(stdout string) (upgradableNames []string, err error) {
-	startRe, _ := regexp.Compile(`The following packages will be upgraded:`)
-	stopRe, _ := regexp.Compile(`^(\d+) upgraded.*`)
+	startRe := regexp.MustCompile(`The following packages will be upgraded:`)
+	stopRe := regexp.MustCompile(`^(\d+) upgraded.*`)
 	startLineFound, stopLineFound := false, false
 
 	lines := strings.Split(stdout, "\n")
@@ -442,9 +399,11 @@ func (o *debian) parseAptGetUpgrade(stdout string) (upgradableNames []string, er
 }
 
 func (o *debian) scanPackageCveInfos(unsecurePacks []models.PackageInfo) (cvePacksList CvePacksList, err error) {
-
-	// { CVE ID: [packageInfo] }
-	cvePackages := make(map[string][]models.PackageInfo)
+	meta := cache.Meta{
+		Name:   o.getServerInfo().GetServerName(),
+		Distro: o.getServerInfo().Distro,
+		Packs:  unsecurePacks,
+	}
 
 	type strarray []string
 	resChan := make(chan struct {
@@ -464,7 +423,6 @@ func (o *debian) scanPackageCveInfos(unsecurePacks []models.PackageInfo) (cvePac
 	}()
 
 	timeout := time.After(30 * 60 * time.Second)
-
 	concurrency := 10
 	tasks := util.GenWorkers(concurrency)
 	for range unsecurePacks {
@@ -472,47 +430,63 @@ func (o *debian) scanPackageCveInfos(unsecurePacks []models.PackageInfo) (cvePac
 			select {
 			case pack := <-reqChan:
 				func(p models.PackageInfo) {
-					if cveIds, err := o.scanPackageCveIds(p); err != nil {
+					changelog := o.getChangelogCache(meta, p)
+					if 0 < len(changelog) {
+						cveIDs := o.getCveIDFromChangelog(changelog, p.Name, p.Version)
+						resChan <- struct {
+							models.PackageInfo
+							strarray
+						}{p, cveIDs}
+						return
+					}
+
+					// if the changelog is not in cache or failed to get from local cache,
+					// get the changelog of the package via internet.
+					if cveIDs, err := o.scanPackageCveIDs(p); err != nil {
 						errChan <- err
 					} else {
 						resChan <- struct {
 							models.PackageInfo
 							strarray
-						}{p, cveIds}
+						}{p, cveIDs}
 					}
 				}(pack)
 			}
 		}
 	}
 
+	// { CVE ID: [packageInfo] }
+	cvePackages := make(map[string][]models.PackageInfo)
+	errs := []error{}
 	for i := 0; i < len(unsecurePacks); i++ {
 		select {
 		case pair := <-resChan:
 			pack := pair.PackageInfo
-			cveIds := pair.strarray
-			for _, cveID := range cveIds {
+			cveIDs := pair.strarray
+			for _, cveID := range cveIDs {
 				cvePackages[cveID] = appendPackIfMissing(cvePackages[cveID], pack)
 			}
 			o.log.Infof("(%d/%d) Scanned %s-%s : %s",
-				i+1, len(unsecurePacks), pair.Name, pair.PackageInfo.Version, cveIds)
+				i+1, len(unsecurePacks), pair.Name, pair.PackageInfo.Version, cveIDs)
 		case err := <-errChan:
-			if err != nil {
-				return nil, err
-			}
+			errs = append(errs, err)
 		case <-timeout:
-			return nil, fmt.Errorf("Timeout scanPackageCveIds")
+			//TODO append to errs
+			return nil, fmt.Errorf("Timeout scanPackageCveIDs")
 		}
 	}
-
-	var cveIds []string
-	for k := range cvePackages {
-		cveIds = append(cveIds, k)
+	if 0 < len(errs) {
+		return nil, fmt.Errorf("%v", errs)
 	}
 
-	o.log.Debugf("%d Cves are found. cves: %v", len(cveIds), cveIds)
+	var cveIDs []string
+	for k := range cvePackages {
+		cveIDs = append(cveIDs, k)
+	}
+	o.log.Debugf("%d Cves are found. cves: %v", len(cveIDs), cveIDs)
 
 	o.log.Info("Fetching CVE details...")
-	cveDetails, err := cveapi.CveClient.FetchCveDetails(cveIds)
+	cveDetails, err := cveapi.CveClient.FetchCveDetails(cveIDs)
 	if err != nil {
 		return nil, err
 	}
@@ -526,13 +500,35 @@ func (o *debian) scanPackageCveInfos(unsecurePacks []models.PackageInfo) (cvePac
 			//  CvssScore: cinfo.CvssScore(conf.Lang),
 		})
 	}
-	sort.Sort(CvePacksList(cvePacksList))
 	return
 }
 
-func (o *debian) scanPackageCveIds(pack models.PackageInfo) (cveIds []string, err error) {
+func (o *debian) getChangelogCache(meta cache.Meta, pack models.PackageInfo) string {
+	cachedPack, found := meta.FindPack(pack.Name)
+	if !found {
+		return ""
+	}
+	if cachedPack.NewVersion != pack.NewVersion {
+		return ""
+	}
+	changelog, err := cache.DB.GetChangelog(meta.Name, pack.Name)
+	if err != nil {
+		o.log.Warnf("Failed to get changelog. bucket: %s, key:%s, err: %s",
+			meta.Name, pack.Name, err)
+		return ""
+	}
+	if len(changelog) == 0 {
+		return ""
+	}
+
+	o.log.Debugf("Cache hit: %s, len: %d, %s...",
+		meta.Name, len(changelog), util.Truncate(changelog, 30))
+	return changelog
+}
+
+func (o *debian) scanPackageCveIDs(pack models.PackageInfo) ([]string, error) {
 	cmd := ""
-	switch o.Family {
+	switch o.Distro.Family {
 	case "ubuntu":
 		cmd = fmt.Sprintf(`apt-get changelog %s | grep '\(urgency\|CVE\)'`, pack.Name)
 	case "debian":
@@ -542,47 +538,45 @@ func (o *debian) scanPackageCveIds(pack models.PackageInfo) (cveIds []string, er
 
 	r := o.ssh(cmd, noSudo)
 	if !r.isSuccess() {
-		o.log.Warnf(
-			"Failed to %s. status: %d, stdout: %s, stderr: %s",
-			cmd, r.ExitStatus, r.Stdout, r.Stderr)
+		o.log.Warnf("Failed to SSH: %s", r)
 		// Ignore this Error.
 		return nil, nil
+	}
 
+	if 0 < len(strings.TrimSpace(r.Stdout)) {
+		err := cache.DB.PutChangelog(o.getServerInfo().GetServerName(), pack.Name, r.Stdout)
+		if err != nil {
+			return nil, fmt.Errorf("Failed to put changelog into cache")
+		}
 	}
-	cveIds, err = o.getCveIDParsingChangelog(r.Stdout, pack.Name, pack.Version)
-	if err != nil {
-		trimUbuntu := strings.Split(pack.Version, "ubuntu")[0]
-		return o.getCveIDParsingChangelog(r.Stdout, pack.Name, trimUbuntu)
-	}
-	return
+	// No error will be returned. Only logging.
+	return o.getCveIDFromChangelog(r.Stdout, pack.Name, pack.Version), nil
 }
 
-func (o *debian) getCveIDParsingChangelog(changelog string,
-	packName string, versionOrLater string) (cveIDs []string, err error) {
+func (o *debian) getCveIDFromChangelog(changelog string,
+	packName string, versionOrLater string) []string {
 
-	cveIDs, err = o.parseChangelog(changelog, packName, versionOrLater)
-	if err == nil {
-		return
+	if cveIDs, err := o.parseChangelog(changelog, packName, versionOrLater); err == nil {
+		return cveIDs
 	}
 
 	ver := strings.Split(versionOrLater, "ubuntu")[0]
-	cveIDs, err = o.parseChangelog(changelog, packName, ver)
-	if err == nil {
-		return
+	if cveIDs, err := o.parseChangelog(changelog, packName, ver); err == nil {
+		return cveIDs
 	}
 
 	splittedByColon := strings.Split(versionOrLater, ":")
 	if 1 < len(splittedByColon) {
 		ver = splittedByColon[1]
 	}
-	cveIDs, err = o.parseChangelog(changelog, packName, ver)
+	cveIDs, err := o.parseChangelog(changelog, packName, ver)
 	if err == nil {
-		return
+		return cveIDs
 	}
 
-	//TODO report as unable to parse changelog.
-	o.log.Warn(err)
-	return []string{}, nil
+	// Only logging the error.
+	o.log.Error(err)
+	return []string{}
 }
 
 // Collect CVE-IDs included in the changelog.
@@ -590,8 +584,8 @@ func (o *debian) getCveIDParsingChangelog(changelog string,
 func (o *debian) parseChangelog(changelog string,
 	packName string, versionOrLater string) (cveIDs []string, err error) {
 
-	cveRe, _ := regexp.Compile(`(CVE-\d{4}-\d{4})`)
-	stopRe, _ := regexp.Compile(fmt.Sprintf(`\(%s\)`, regexp.QuoteMeta(versionOrLater)))
+	cveRe := regexp.MustCompile(`(CVE-\d{4}-\d{4,})`)
+	stopRe := regexp.MustCompile(fmt.Sprintf(`\(%s\)`, regexp.QuoteMeta(versionOrLater)))
 	stopLineFound := false
 	lines := strings.Split(changelog, "\n")
 	for _, line := range lines {
@@ -599,7 +593,7 @@ func (o *debian) parseChangelog(changelog string,
 			o.log.Debugf("Found the stop line. line: %s", line)
 			stopLineFound = true
 			break
-		} else if matches := cveRe.FindAllString(line, -1); len(matches) > 0 {
+		} else if matches := cveRe.FindAllString(line, -1); 0 < len(matches) {
 			for _, m := range matches {
 				cveIDs = util.AppendIfMissing(cveIDs, m)
 			}
@@ -615,6 +609,29 @@ func (o *debian) parseChangelog(changelog string,
 	return
 }
 
+func (o *debian) splitAptCachePolicy(stdout string) map[string]string {
+	//  re := regexp.MustCompile(`(?m:^[^ \t]+:$)`)
+	re := regexp.MustCompile(`(?m:^[^ \t]+:\r\n)`)
+	ii := re.FindAllStringIndex(stdout, -1)
+	ri := []int{}
+	for i := len(ii) - 1; 0 <= i; i-- {
+		ri = append(ri, ii[i][0])
+	}
+	splitted := []string{}
+	lasti := len(stdout)
+	for _, i := range ri {
+		splitted = append(splitted, stdout[i:lasti])
+		lasti = i
+	}
+
+	packChangelog := map[string]string{}
+	for _, r := range splitted {
+		packName := r[:strings.Index(r, ":")]
+		packChangelog[packName] = r
+	}
+	return packChangelog
+}
+
 type packCandidateVer struct {
 	Name      string
 	Installed string

scan/debian_test.go

@@ -18,14 +18,18 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 package scan
 
 import (
+	"os"
 	"reflect"
 	"testing"
 
+	"github.com/Sirupsen/logrus"
+	"github.com/future-architect/vuls/cache"
 	"github.com/future-architect/vuls/config"
+	"github.com/future-architect/vuls/models"
 	"github.com/k0kubun/pp"
 )
 
-func TestParseScanedPackagesLineDebian(t *testing.T) {
+func TestParseScannedPackagesLineDebian(t *testing.T) {
 
 	var packagetests = []struct {
 		in      string
@@ -43,7 +47,7 @@ func TestParseScanedPackagesLineDebian(t *testing.T) {
 
 	d := newDebian(config.ServerInfo{})
 	for _, tt := range packagetests {
-		n, v, _ := d.parseScanedPackagesLine(tt.in)
+		n, v, _ := d.parseScannedPackagesLine(tt.in)
 		if n != tt.name {
 			t.Errorf("name: expected %s, actual %s", tt.name, n)
 		}
@@ -54,7 +58,7 @@ func TestParseScanedPackagesLineDebian(t *testing.T) {
 
 }
 
-func TestgetCveIDParsingChangelog(t *testing.T) {
+func TestGetCveIDParsingChangelog(t *testing.T) {
 
 	var tests = []struct {
 		in       []string
@@ -86,12 +90,11 @@ systemd (227-1) unstable; urgency=medium`,
 				"CVE-2015-3210",
 			},
 		},
-
 		{
 			// ver
 			[]string{
 				"libpcre3",
-				"2:8.38-1ubuntu1",
+				"2:8.35-7.1ubuntu1",
 				`pcre3 (2:8.38-2) unstable; urgency=low
 pcre3 (2:8.38-1) unstable; urgency=low
 pcre3 (2:8.35-8) unstable; urgency=low
@@ -110,7 +113,6 @@ pcre3 (2:8.35-7) unstable; urgency=medium`,
 				"CVE-2015-3210",
 			},
 		},
-
 		{
 			// ver-ubuntu3
 			[]string{
@@ -151,7 +153,7 @@ sysvinit (2.88dsf-57) unstable; urgency=low`,
 util-linux (2.27.1-3) unstable; urgency=medium
 CVE-2015-2325: heap buffer overflow in compile_branch(). (Closes: #781795)
 CVE-2015-2326: heap buffer overflow in pcre_compile2(). (Closes: #783285)
-CVE-2015-3210: heap buffer overflow in pcre_compile2() /
+CVE-2015-3210: CVE-2016-1000000heap buffer overflow in pcre_compile2() /
 util-linux (2.27.1-2) unstable; urgency=medium
 util-linux (2.27.1-1ubuntu4) xenial; urgency=medium
 util-linux (2.27.1-1ubuntu3) xenial; urgency=medium
@@ -178,15 +180,17 @@ util-linux (2.26.2-6) unstable; urgency=medium`,
 				"CVE-2015-2325",
 				"CVE-2015-2326",
 				"CVE-2015-3210",
+				"CVE-2016-1000000",
 			},
 		},
 	}
 
 	d := newDebian(config.ServerInfo{})
 	for _, tt := range tests {
-		actual, _ := d.getCveIDParsingChangelog(tt.in[2], tt.in[0], tt.in[1])
+		actual := d.getCveIDFromChangelog(tt.in[2], tt.in[0], tt.in[1])
 		if len(actual) != len(tt.expected) {
 			t.Errorf("Len of return array are'nt same. expected %#v, actual %#v", tt.expected, actual)
+			t.Errorf(pp.Sprintf("%s", tt.in))
 			continue
 		}
 		for i := range tt.expected {
@@ -195,13 +199,6 @@ util-linux (2.26.2-6) unstable; urgency=medium`,
 			}
 		}
 	}
-
-	for _, tt := range tests {
-		_, err := d.getCveIDParsingChangelog(tt.in[2], tt.in[0], "version number do'nt match case")
-		if err != nil {
-			t.Errorf("Returning error is unexpected")
-		}
-	}
 }
 
 func TestGetUpdatablePackNames(t *testing.T) {
@@ -520,6 +517,95 @@ Calculating upgrade... Done
 	}
 }
 
+func TestGetChangelogCache(t *testing.T) {
+	const servername = "server1"
+	pack := models.PackageInfo{
+		Name:       "apt",
+		Version:    "1.0.0",
+		NewVersion: "1.0.1",
+	}
+	var meta = cache.Meta{
+		Name: servername,
+		Distro: config.Distro{
+			Family:  "ubuntu",
+			Release: "16.04",
+		},
+		Packs: []models.PackageInfo{pack},
+	}
+
+	const path = "/tmp/vuls-test-cache-11111111.db"
+	log := logrus.NewEntry(&logrus.Logger{})
+	if err := cache.SetupBolt(path, log); err != nil {
+		t.Errorf("Failed to setup bolt: %s", err)
+	}
+	defer os.Remove(path)
+
+	if err := cache.DB.EnsureBuckets(meta); err != nil {
+		t.Errorf("Failed to ensure buckets: %s", err)
+	}
+
+	d := newDebian(config.ServerInfo{})
+	actual := d.getChangelogCache(meta, pack)
+	if actual != "" {
+		t.Errorf("Failed to get empty stirng from cache:")
+	}
+
+	clog := "changelog-text"
+	if err := cache.DB.PutChangelog(servername, "apt", clog); err != nil {
+		t.Errorf("Failed to put changelog: %s", err)
+	}
+
+	actual = d.getChangelogCache(meta, pack)
+	if actual != clog {
+		t.Errorf("Failed to get changelog from cache: %s", actual)
+	}
+
+	// increment a version of the pack
+	pack.NewVersion = "1.0.2"
+	actual = d.getChangelogCache(meta, pack)
+	if actual != "" {
+		t.Errorf("The changelog is not invalidated: %s", actual)
+	}
+
+	// change a name of the pack
+	pack.Name = "bash"
+	actual = d.getChangelogCache(meta, pack)
+	if actual != "" {
+		t.Errorf("The changelog is not invalidated: %s", actual)
+	}
+}
+
+func TestSplitAptCachePolicy(t *testing.T) {
+	var tests = []struct {
+		stdout   string
+		expected map[string]string
+	}{
+		// This function parse apt-cache policy by using Regexp multi-line mode.
+		// So, test data includes "\r\n"
+		{
+			"apt:\r\n  Installed: 1.2.6\r\n  Candidate: 1.2.12~ubuntu16.04.1\r\n  Version table:\r\n     1.2.12~ubuntu16.04.1 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages\r\n     1.2.10ubuntu1 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages\r\n *** 1.2.6 100\r\n        100 /var/lib/dpkg/status\r\napt-utils:\r\n  Installed: 1.2.6\r\n  Candidate: 1.2.12~ubuntu16.04.1\r\n  Version table:\r\n     1.2.12~ubuntu16.04.1 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages\r\n     1.2.10ubuntu1 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages\r\n *** 1.2.6 100\r\n        100 /var/lib/dpkg/status\r\nbase-files:\r\n  Installed: 9.4ubuntu3\r\n  Candidate: 9.4ubuntu4.2\r\n  Version table:\r\n     9.4ubuntu4.2 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages\r\n     9.4ubuntu4 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages\r\n *** 9.4ubuntu3 100\r\n        100 /var/lib/dpkg/status\r\n",
+
+			map[string]string{
+				"apt": "apt:\r\n  Installed: 1.2.6\r\n  Candidate: 1.2.12~ubuntu16.04.1\r\n  Version table:\r\n     1.2.12~ubuntu16.04.1 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages\r\n     1.2.10ubuntu1 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages\r\n *** 1.2.6 100\r\n        100 /var/lib/dpkg/status\r\n",
+
+				"apt-utils": "apt-utils:\r\n  Installed: 1.2.6\r\n  Candidate: 1.2.12~ubuntu16.04.1\r\n  Version table:\r\n     1.2.12~ubuntu16.04.1 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages\r\n     1.2.10ubuntu1 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages\r\n *** 1.2.6 100\r\n        100 /var/lib/dpkg/status\r\n",
+
+				"base-files": "base-files:\r\n  Installed: 9.4ubuntu3\r\n  Candidate: 9.4ubuntu4.2\r\n  Version table:\r\n     9.4ubuntu4.2 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages\r\n     9.4ubuntu4 500\r\n        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages\r\n *** 9.4ubuntu3 100\r\n        100 /var/lib/dpkg/status\r\n",
+			},
+		},
+	}
+
+	d := newDebian(config.ServerInfo{})
+	for _, tt := range tests {
+		actual := d.splitAptCachePolicy(tt.stdout)
+		if !reflect.DeepEqual(tt.expected, actual) {
+			e := pp.Sprintf("%v", tt.expected)
+			a := pp.Sprintf("%v", actual)
+			t.Errorf("expected %s, actual %s", e, a)
+		}
+	}
+}
+
 func TestParseAptCachePolicy(t *testing.T) {
 
 	var tests = []struct {

scan/freebsd.go

@@ -0,0 +1,263 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package scan
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/future-architect/vuls/config"
+	"github.com/future-architect/vuls/cveapi"
+	"github.com/future-architect/vuls/models"
+	"github.com/future-architect/vuls/util"
+)
+
+// inherit OsTypeInterface
+type bsd struct {
+	base
+}
+
+// NewBSD constructor
+func newBsd(c config.ServerInfo) *bsd {
+	d := &bsd{}
+	d.log = util.NewCustomLogger(c)
+	d.setServerInfo(c)
+	return d
+}
+
+//https://github.com/mizzy/specinfra/blob/master/lib/specinfra/helper/detect_os/freebsd.rb
+func detectFreebsd(c config.ServerInfo) (itsMe bool, bsd osTypeInterface) {
+	bsd = newBsd(c)
+
+	// Prevent from adding `set -o pipefail` option
+	c.Distro = config.Distro{Family: "FreeBSD"}
+
+	if r := sshExec(c, "uname", noSudo); r.isSuccess() {
+		if strings.Contains(r.Stdout, "FreeBSD") == true {
+			if b := sshExec(c, "uname -r", noSudo); b.isSuccess() {
+				rel := strings.TrimSpace(b.Stdout)
+				bsd.setDistro("FreeBSD", rel)
+				return true, bsd
+			}
+		}
+	}
+	Log.Debugf("Not FreeBSD. servernam: %s", c.ServerName)
+	return false, bsd
+}
+
+func (o *bsd) checkIfSudoNoPasswd() error {
+	// FreeBSD doesn't need root privilege
+	o.log.Infof("sudo ... OK")
+	return nil
+}
+
+func (o *bsd) checkDependencies() error {
+	return nil
+}
+
+func (o *bsd) install() error {
+	return nil
+}
+
+func (o *bsd) checkRequiredPackagesInstalled() error {
+	return nil
+}
+
+func (o *bsd) scanPackages() error {
+	var err error
+	var packs []models.PackageInfo
+	if packs, err = o.scanInstalledPackages(); err != nil {
+		o.log.Errorf("Failed to scan installed packages")
+		return err
+	}
+	o.setPackages(packs)
+
+	var unsecurePacks []CvePacksInfo
+	if unsecurePacks, err = o.scanUnsecurePackages(); err != nil {
+		o.log.Errorf("Failed to scan vulnerable packages")
+		return err
+	}
+	o.setUnsecurePackages(unsecurePacks)
+	return nil
+}
+
+func (o *bsd) scanInstalledPackages() ([]models.PackageInfo, error) {
+	cmd := util.PrependProxyEnv("pkg version -v")
+	r := o.ssh(cmd, noSudo)
+	if !r.isSuccess() {
+		return nil, fmt.Errorf("Failed to SSH: %s", r)
+	}
+	return o.parsePkgVersion(r.Stdout), nil
+}
+
+func (o *bsd) scanUnsecurePackages() (cvePacksList []CvePacksInfo, err error) {
+	const vulndbPath = "/tmp/vuln.db"
+	cmd := "rm -f " + vulndbPath
+	r := o.ssh(cmd, noSudo)
+	if !r.isSuccess(0) {
+		return nil, fmt.Errorf("Failed to SSH: %s", r)
+	}
+
+	cmd = util.PrependProxyEnv("pkg audit -F -r -f " + vulndbPath)
+	r = o.ssh(cmd, noSudo)
+	if !r.isSuccess(0, 1) {
+		return nil, fmt.Errorf("Failed to SSH: %s", r)
+	}
+	if r.ExitStatus == 0 {
+		// no vulnerabilities
+		return []CvePacksInfo{}, nil
+	}
+
+	var packAdtRslt []pkgAuditResult
+	blocks := o.splitIntoBlocks(r.Stdout)
+	for _, b := range blocks {
+		name, cveIDs, vulnID := o.parseBlock(b)
+		if len(cveIDs) == 0 {
+			continue
+		}
+		pack, found := o.Packages.FindByName(name)
+		if !found {
+			return nil, fmt.Errorf("Vulnerable package: %s is not found", name)
+		}
+		packAdtRslt = append(packAdtRslt, pkgAuditResult{
+			pack: pack,
+			vulnIDCveIDs: vulnIDCveIDs{
+				vulnID: vulnID,
+				cveIDs: cveIDs,
+			},
+		})
+	}
+
+	// { CVE ID: []pkgAuditResult }
+	cveIDAdtMap := make(map[string][]pkgAuditResult)
+	for _, p := range packAdtRslt {
+		for _, cid := range p.vulnIDCveIDs.cveIDs {
+			cveIDAdtMap[cid] = append(cveIDAdtMap[cid], p)
+		}
+	}
+
+	cveIDs := []string{}
+	for k := range cveIDAdtMap {
+		cveIDs = append(cveIDs, k)
+	}
+
+	cveDetails, err := cveapi.CveClient.FetchCveDetails(cveIDs)
+	if err != nil {
+		return nil, err
+	}
+	o.log.Info("Done")
+
+	for _, d := range cveDetails {
+		packs := []models.PackageInfo{}
+		for _, r := range cveIDAdtMap[d.CveID] {
+			packs = append(packs, r.pack)
+		}
+
+		disAdvs := []models.DistroAdvisory{}
+		for _, r := range cveIDAdtMap[d.CveID] {
+			disAdvs = append(disAdvs, models.DistroAdvisory{
+				AdvisoryID: r.vulnIDCveIDs.vulnID,
+			})
+		}
+
+		cvePacksList = append(cvePacksList, CvePacksInfo{
+			CveID:            d.CveID,
+			CveDetail:        d,
+			Packs:            packs,
+			DistroAdvisories: disAdvs,
+		})
+	}
+	return
+}
+
+func (o *bsd) parsePkgVersion(stdout string) (packs []models.PackageInfo) {
+	lines := strings.Split(stdout, "\n")
+	for _, l := range lines {
+		fields := strings.Fields(l)
+		if len(fields) < 2 {
+			continue
+		}
+
+		packVer := fields[0]
+		splitted := strings.Split(packVer, "-")
+		ver := splitted[len(splitted)-1]
+		name := strings.Join(splitted[:len(splitted)-1], "-")
+
+		switch fields[1] {
+		case "?", "=":
+			packs = append(packs, models.PackageInfo{
+				Name:    name,
+				Version: ver,
+			})
+		case "<":
+			candidate := strings.TrimSuffix(fields[6], ")")
+			packs = append(packs, models.PackageInfo{
+				Name:       name,
+				Version:    ver,
+				NewVersion: candidate,
+			})
+		}
+	}
+	return
+}
+
+type vulnIDCveIDs struct {
+	vulnID string
+	cveIDs []string
+}
+
+type pkgAuditResult struct {
+	pack         models.PackageInfo
+	vulnIDCveIDs vulnIDCveIDs
+}
+
+func (o *bsd) splitIntoBlocks(stdout string) (blocks []string) {
+	lines := strings.Split(stdout, "\n")
+	block := []string{}
+	for _, l := range lines {
+		if len(strings.TrimSpace(l)) == 0 {
+			if 0 < len(block) {
+				blocks = append(blocks, strings.Join(block, "\n"))
+				block = []string{}
+			}
+			continue
+		}
+		block = append(block, strings.TrimSpace(l))
+	}
+	if 0 < len(block) {
+		blocks = append(blocks, strings.Join(block, "\n"))
+	}
+	return
+}
+
+func (o *bsd) parseBlock(block string) (packName string, cveIDs []string, vulnID string) {
+	lines := strings.Split(block, "\n")
+	for _, l := range lines {
+		if strings.HasSuffix(l, " is vulnerable:") {
+			packVer := strings.Fields(l)[0]
+			splitted := strings.Split(packVer, "-")
+			packName = strings.Join(splitted[:len(splitted)-1], "-")
+		} else if strings.HasPrefix(l, "CVE:") {
+			cveIDs = append(cveIDs, strings.Fields(l)[1])
+		} else if strings.HasPrefix(l, "WWW:") {
+			splitted := strings.Split(l, "/")
+			vulnID = strings.TrimSuffix(splitted[len(splitted)-1], ".html")
+		}
+	}
+	return
+}

scan/freebsd_test.go

@@ -0,0 +1,155 @@
+package scan
+
+import (
+	"reflect"
+	"testing"
+
+	"github.com/future-architect/vuls/config"
+	"github.com/future-architect/vuls/models"
+	"github.com/k0kubun/pp"
+)
+
+func TestParsePkgVersion(t *testing.T) {
+	var tests = []struct {
+		in       string
+		expected []models.PackageInfo
+	}{
+		{
+			`Updating FreeBSD repository catalogue...
+FreeBSD repository is up-to-date.
+All repositories are up-to-date.
+bash-4.2.45                        <   needs updating (remote has 4.3.42_1)
+gettext-0.18.3.1                   <   needs updating (remote has 0.19.7)
+tcl84-8.4.20_2,1                   =   up-to-date with remote
+teTeX-base-3.0_25                  ?   orphaned: print/teTeX-base`,
+
+			[]models.PackageInfo{
+				{
+					Name:       "bash",
+					Version:    "4.2.45",
+					NewVersion: "4.3.42_1",
+				},
+				{
+					Name:       "gettext",
+					Version:    "0.18.3.1",
+					NewVersion: "0.19.7",
+				},
+				{
+					Name:    "tcl84",
+					Version: "8.4.20_2,1",
+				},
+				{
+					Name:    "teTeX-base",
+					Version: "3.0_25",
+				},
+			},
+		},
+	}
+
+	d := newBsd(config.ServerInfo{})
+	for _, tt := range tests {
+		actual := d.parsePkgVersion(tt.in)
+		if !reflect.DeepEqual(tt.expected, actual) {
+			e := pp.Sprintf("%v", tt.expected)
+			a := pp.Sprintf("%v", actual)
+			t.Errorf("expected %s, actual %s", e, a)
+		}
+	}
+}
+
+func TestSplitIntoBlocks(t *testing.T) {
+	var tests = []struct {
+		in       string
+		expected []string
+	}{
+		{
+			`
+block1
+
+block2
+block2
+block2
+
+block3
+block3`,
+			[]string{
+				`block1`,
+				"block2\nblock2\nblock2",
+				"block3\nblock3",
+			},
+		},
+	}
+
+	d := newBsd(config.ServerInfo{})
+	for _, tt := range tests {
+		actual := d.splitIntoBlocks(tt.in)
+		if !reflect.DeepEqual(tt.expected, actual) {
+			e := pp.Sprintf("%v", tt.expected)
+			a := pp.Sprintf("%v", actual)
+			t.Errorf("expected %s, actual %s", e, a)
+		}
+	}
+
+}
+
+func TestParseBlock(t *testing.T) {
+	var tests = []struct {
+		in     string
+		name   string
+		cveIDs []string
+		vulnID string
+	}{
+		{
+
+			in: `vulnxml file up-to-date
+bind96-9.6.3.2.ESV.R10_2 is vulnerable:
+bind -- denial of service vulnerability
+CVE: CVE-2014-0591
+WWW: https://vuxml.FreeBSD.org/freebsd/cb252f01-7c43-11e3-b0a6-005056a37f68.html`,
+			name:   "bind96",
+			cveIDs: []string{"CVE-2014-0591"},
+			vulnID: "cb252f01-7c43-11e3-b0a6-005056a37f68",
+		},
+		{
+			in: `bind96-9.6.3.2.ESV.R10_2 is vulnerable:
+bind -- denial of service vulnerability
+CVE: CVE-2014-8680
+CVE: CVE-2014-8500
+WWW: https://vuxml.FreeBSD.org/freebsd/ab3e98d9-8175-11e4-907d-d050992ecde8.html`,
+			name:   "bind96",
+			cveIDs: []string{"CVE-2014-8680", "CVE-2014-8500"},
+			vulnID: "ab3e98d9-8175-11e4-907d-d050992ecde8",
+		},
+		{
+			in: `hoge-hoge-9.6.3.2.ESV.R10_2 is vulnerable:
+bind -- denial of service vulnerability
+CVE: CVE-2014-8680
+CVE: CVE-2014-8500
+WWW: https://vuxml.FreeBSD.org/freebsd/ab3e98d9-8175-11e4-907d-d050992ecde8.html`,
+			name:   "hoge-hoge",
+			cveIDs: []string{"CVE-2014-8680", "CVE-2014-8500"},
+			vulnID: "ab3e98d9-8175-11e4-907d-d050992ecde8",
+		},
+		{
+			in:     `1 problem(s) in the installed packages found.`,
+			cveIDs: []string{},
+			vulnID: "",
+		},
+	}
+
+	d := newBsd(config.ServerInfo{})
+	for _, tt := range tests {
+		aName, aCveIDs, aVulnID := d.parseBlock(tt.in)
+		if tt.name != aName {
+			t.Errorf("expected vulnID: %s, actual %s", tt.vulnID, aVulnID)
+		}
+		for i := range tt.cveIDs {
+			if tt.cveIDs[i] != aCveIDs[i] {
+				t.Errorf("expected cveID: %s, actual %s", tt.cveIDs[i], aCveIDs[i])
+			}
+		}
+		if tt.vulnID != aVulnID {
+			t.Errorf("expected vulnID: %s, actual %s", tt.vulnID, aVulnID)
+		}
+	}
+}

scan/linux.go

@@ -1,134 +0,0 @@
-/* Vuls - Vulnerability Scanner
-Copyright (C) 2016  Future Architect, Inc. Japan.
-
-This program is free software: you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation, either version 3 of the License, or
-(at your option) any later version.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program.  If not, see <http://www.gnu.org/licenses/>.
-*/
-
-package scan
-
-import (
-	"fmt"
-	"sort"
-
-	"github.com/Sirupsen/logrus"
-	"github.com/future-architect/vuls/config"
-	"github.com/future-architect/vuls/cveapi"
-	"github.com/future-architect/vuls/models"
-)
-
-type linux struct {
-	ServerInfo config.ServerInfo
-
-	Family  string
-	Release string
-	osPackages
-	log *logrus.Entry
-}
-
-func (l *linux) ssh(cmd string, sudo bool) sshResult {
-	return sshExec(l.ServerInfo, cmd, sudo, l.log)
-}
-
-func (l *linux) setServerInfo(c config.ServerInfo) {
-	l.ServerInfo = c
-}
-
-func (l *linux) getServerInfo() config.ServerInfo {
-	return l.ServerInfo
-}
-
-func (l *linux) setDistributionInfo(fam, rel string) {
-	l.Family = fam
-	l.Release = rel
-}
-
-func (l *linux) getDistributionInfo() string {
-	return fmt.Sprintf("%s %s", l.Family, l.Release)
-}
-
-func (l *linux) convertToModel() (models.ScanResult, error) {
-	var cves, unknownScoreCves []models.CveInfo
-	for _, p := range l.UnsecurePackages {
-		if p.CveDetail.CvssScore(config.Conf.Lang) < 0 {
-			unknownScoreCves = append(unknownScoreCves, models.CveInfo{
-				CveDetail:        p.CveDetail,
-				Packages:         p.Packs,
-				DistroAdvisories: p.DistroAdvisories, // only Amazon Linux
-			})
-			continue
-		}
-
-		cpenames := []models.CpeName{}
-		for _, cpename := range p.CpeNames {
-			cpenames = append(cpenames,
-				models.CpeName{Name: cpename})
-		}
-
-		cve := models.CveInfo{
-			CveDetail:        p.CveDetail,
-			Packages:         p.Packs,
-			DistroAdvisories: p.DistroAdvisories, // only Amazon Linux
-			CpeNames:         cpenames,
-		}
-		cves = append(cves, cve)
-	}
-
-	return models.ScanResult{
-		ServerName:  l.ServerInfo.ServerName,
-		Family:      l.Family,
-		Release:     l.Release,
-		KnownCves:   cves,
-		UnknownCves: unknownScoreCves,
-	}, nil
-}
-
-// scanVulnByCpeName search vulnerabilities that specified in config file.
-func (l *linux) scanVulnByCpeName() error {
-	unsecurePacks := CvePacksList{}
-
-	serverInfo := l.getServerInfo()
-	cpeNames := serverInfo.CpeNames
-
-	// remove duplicate
-	set := map[string]CvePacksInfo{}
-
-	for _, name := range cpeNames {
-		details, err := cveapi.CveClient.FetchCveDetailsByCpeName(name)
-		if err != nil {
-			return err
-		}
-		for _, detail := range details {
-			if val, ok := set[detail.CveID]; ok {
-				names := val.CpeNames
-				names = append(names, name)
-				val.CpeNames = names
-				set[detail.CveID] = val
-			} else {
-				set[detail.CveID] = CvePacksInfo{
-					CveID:     detail.CveID,
-					CveDetail: detail,
-					CpeNames:  []string{name},
-				}
-			}
-		}
-	}
-
-	for key := range set {
-		unsecurePacks = append(unsecurePacks, set[key])
-	}
-	unsecurePacks = append(unsecurePacks, l.UnsecurePackages...)
-	sort.Sort(CvePacksList(unsecurePacks))
-	l.setUnsecurePackages(unsecurePacks)
-	return nil
-}

scan/linux_test.go

@@ -1,18 +0,0 @@
-/* Vuls - Vulnerability Scanner
-Copyright (C) 2016  Future Architect, Inc. Japan.
-
-This program is free software: you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation, either version 3 of the License, or
-(at your option) any later version.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program.  If not, see <http://www.gnu.org/licenses/>.
-*/
-
-package scan

scan/redhat.go

@@ -35,28 +35,24 @@ import (
 
 // inherit OsTypeInterface
 type redhat struct {
-	linux
+	base
 }
 
 // NewRedhat is constructor
 func newRedhat(c config.ServerInfo) *redhat {
 	r := &redhat{}
 	r.log = util.NewCustomLogger(c)
+	r.setServerInfo(c)
 	return r
 }
 
 // https://github.com/serverspec/specinfra/blob/master/lib/specinfra/helper/detect_os/redhat.rb
 func detectRedhat(c config.ServerInfo) (itsMe bool, red osTypeInterface) {
-
 	red = newRedhat(c)
 
-	// set sudo option flag
-	c.SudoOpt = config.SudoOption{ExecBySudo: true}
-	red.setServerInfo(c)
-
 	if r := sshExec(c, "ls /etc/fedora-release", noSudo); r.isSuccess() {
-		red.setDistributionInfo("fedora", "unknown")
-		Log.Warn("Fedora not tested yet. Host: %s:%s", c.Host, c.Port)
+		red.setDistro("fedora", "unknown")
+		Log.Warn("Fedora not tested yet: %s", r)
 		return true, red
 	}
 
@@ -66,21 +62,19 @@ func detectRedhat(c config.ServerInfo) (itsMe bool, red osTypeInterface) {
 		// $ cat /etc/redhat-release
 		// CentOS release 6.5 (Final)
 		if r := sshExec(c, "cat /etc/redhat-release", noSudo); r.isSuccess() {
-			re, _ := regexp.Compile(`(.*) release (\d[\d.]*)`)
+			re := regexp.MustCompile(`(.*) release (\d[\d.]*)`)
 			result := re.FindStringSubmatch(strings.TrimSpace(r.Stdout))
 			if len(result) != 3 {
-				Log.Warn(
-					"Failed to parse RedHat/CentOS version. stdout: %s, Host: %s:%s",
-					r.Stdout, c.Host, c.Port)
+				Log.Warn("Failed to parse RedHat/CentOS version: %s", r)
 				return true, red
 			}
 
 			release := result[2]
 			switch strings.ToLower(result[1]) {
 			case "centos", "centos linux":
-				red.setDistributionInfo("centos", release)
+				red.setDistro("centos", release)
 			default:
-				red.setDistributionInfo("rhel", release)
+				red.setDistro("rhel", release)
 			}
 			return true, red
 		}
@@ -96,116 +90,84 @@ func detectRedhat(c config.ServerInfo) (itsMe bool, red osTypeInterface) {
 				release = fields[4]
 			}
 		}
-		red.setDistributionInfo(family, release)
+		red.setDistro(family, release)
 		return true, red
 	}
 
-	Log.Debugf("Not RedHat like Linux. Host: %s:%s", c.Host, c.Port)
+	Log.Debugf("Not RedHat like Linux. servername: %s", c.ServerName)
 	return false, red
 }
 
-// CentOS 5 ... yum-plugin-security, yum-changelog
-// CentOS 6 ... yum-plugin-security, yum-plugin-changelog
-// CentOS 7 ... yum-plugin-security, yum-plugin-changelog
-// RHEL, Amazon ... no additinal packages needed
-func (o *redhat) install() error {
-
-	switch o.Family {
-	case "rhel", "amazon":
-		o.log.Infof("Nothing to do")
-		return nil
-	}
-
-	if err := o.installYumPluginSecurity(); err != nil {
-		return err
-	}
-	return o.installYumChangelog()
-}
-
-func (o *redhat) installYumPluginSecurity() error {
-
-	if r := o.ssh("rpm -q yum-plugin-security", noSudo); r.isSuccess() {
-		o.log.Infof("Ignored: yum-plugin-security already installed")
-		return nil
-	}
-
-	o.log.Info("Installing yum-plugin-security...")
-	cmd := util.PrependProxyEnv("yum install -y yum-plugin-security")
-	if r := o.ssh(cmd, sudo); !r.isSuccess() {
-		return fmt.Errorf(
-			"Failed to %s. status: %d, stdout: %s, stderr: %s",
-			cmd, r.ExitStatus, r.Stdout, r.Stderr)
+func (o *redhat) checkIfSudoNoPasswd() error {
+	r := o.ssh("yum --version", o.sudo())
+	if !r.isSuccess() {
+		o.log.Errorf("sudo error on %s", r)
+		return fmt.Errorf("Failed to sudo: %s", r)
 	}
+	o.log.Infof("sudo ... OK")
 	return nil
 }
 
-func (o *redhat) installYumChangelog() error {
+// CentOS 5 ... yum-changelog
+// CentOS 6 ... yum-plugin-changelog
+// CentOS 7 ... yum-plugin-changelog
+// RHEL, Amazon ... no additinal packages needed
+func (o *redhat) checkDependencies() error {
+	switch o.Distro.Family {
+	case "rhel", "amazon":
+		//  o.log.Infof("Nothing to do")
+		return nil
 
-	if o.Family == "centos" {
+	case "centos":
 		var majorVersion int
-		if 0 < len(o.Release) {
-			majorVersion, _ = strconv.Atoi(strings.Split(o.Release, ".")[0])
+		if 0 < len(o.Distro.Release) {
+			majorVersion, _ = strconv.Atoi(strings.Split(o.Distro.Release, ".")[0])
 		} else {
-			return fmt.Errorf(
-				"Not implemented yet. family: %s, release: %s",
-				o.Family, o.Release)
+			return fmt.Errorf("Not implemented yet: %s", o.Distro)
 		}
 
-		var packName = ""
+		var name = "yum-plugin-changelog"
 		if majorVersion < 6 {
-			packName = "yum-changelog"
-		} else {
-			packName = "yum-plugin-changelog"
+			name = "yum-changelog"
 		}
 
-		cmd := "rpm -q " + packName
+		cmd := "rpm -q " + name
 		if r := o.ssh(cmd, noSudo); r.isSuccess() {
-			o.log.Infof("Ignored: %s already installed", packName)
 			return nil
 		}
+		o.lackDependencies = []string{name}
+		return nil
 
-		o.log.Infof("Installing %s...", packName)
-		cmd = util.PrependProxyEnv("yum install -y " + packName)
+	default:
+		return fmt.Errorf("Not implemented yet: %s", o.Distro)
+	}
+}
+
+func (o *redhat) install() error {
+	for _, name := range o.lackDependencies {
+		cmd := util.PrependProxyEnv("yum install -y " + name)
 		if r := o.ssh(cmd, sudo); !r.isSuccess() {
-			return fmt.Errorf(
-				"Failed to install %s. status: %d, stdout: %s, stderr: %s",
-				packName, r.ExitStatus, r.Stdout, r.Stderr)
+			return fmt.Errorf("Failed to SSH: %s", r)
 		}
-		o.log.Infof("Installed: %s", packName)
+		o.log.Infof("Installed: %s", name)
 	}
 	return nil
 }
 
 func (o *redhat) checkRequiredPackagesInstalled() error {
-	if config.Conf.UseYumPluginSecurity {
-		// check if yum-plugin-security is installed.
-		// Amazon Linux, REHL can execute 'yum updateinfo --security updates' without yum-plugin-security
-		cmd := "rpm -q yum-plugin-security"
-		if o.Family == "centos" {
-			if r := o.ssh(cmd, noSudo); !r.isSuccess() {
-				msg := "yum-plugin-security is not installed"
-				o.log.Errorf(msg)
-				return fmt.Errorf(msg)
-			}
-		}
-		return nil
-	}
-
-	if o.Family == "centos" {
+	if o.Distro.Family == "centos" {
 		var majorVersion int
-		if 0 < len(o.Release) {
-			majorVersion, _ = strconv.Atoi(strings.Split(o.Release, ".")[0])
+		if 0 < len(o.Distro.Release) {
+			majorVersion, _ = strconv.Atoi(strings.Split(o.Distro.Release, ".")[0])
 		} else {
-			msg := fmt.Sprintf("Not implemented yet. family: %s, release: %s", o.Family, o.Release)
+			msg := fmt.Sprintf("Not implemented yet: %s", o.Distro)
 			o.log.Errorf(msg)
 			return fmt.Errorf(msg)
 		}
 
-		var packName = ""
+		var packName = "yum-plugin-changelog"
 		if majorVersion < 6 {
 			packName = "yum-changelog"
-		} else {
-			packName = "yum-plugin-changelog"
 		}
 
 		cmd := "rpm -q " + packName
@@ -274,7 +236,7 @@ func (o *redhat) parseScannedPackagesLine(line string) (models.PackageInfo, erro
 }
 
 func (o *redhat) scanUnsecurePackages() ([]CvePacksInfo, error) {
-	if o.Family != "centos" || config.Conf.UseYumPluginSecurity {
+	if o.Distro.Family != "centos" {
 		// Amazon, RHEL has yum updateinfo as default
 		// yum updateinfo can collenct vendor advisory information.
 		return o.scanUnsecurePackagesUsingYumPluginSecurity()
@@ -284,16 +246,19 @@ func (o *redhat) scanUnsecurePackages() ([]CvePacksInfo, error) {
 	return o.scanUnsecurePackagesUsingYumCheckUpdate()
 }
 
-//TODO return whether already expired.
+// For CentOS
 func (o *redhat) scanUnsecurePackagesUsingYumCheckUpdate() (CvePacksList, error) {
+	cmd := "LANGUAGE=en_US.UTF-8 yum --color=never %s check-update"
+	if o.getServerInfo().Enablerepo != "" {
+		cmd = fmt.Sprintf(cmd, "--enablerepo="+o.getServerInfo().Enablerepo)
+	} else {
+		cmd = fmt.Sprintf(cmd, "")
+	}
 
-	cmd := "yum check-update"
 	r := o.ssh(util.PrependProxyEnv(cmd), sudo)
 	if !r.isSuccess(0, 100) {
 		//returns an exit code of 100 if there are available updates.
-		return nil, fmt.Errorf(
-			"Failed to %s. status: %d, stdout: %s, stderr: %s",
-			cmd, r.ExitStatus, r.Stdout, r.Stderr)
+		return nil, fmt.Errorf("Failed to SSH: %s", r)
 	}
 
 	// get Updateble package name, installed, candidate version.
@@ -308,13 +273,22 @@ func (o *redhat) scanUnsecurePackagesUsingYumCheckUpdate() (CvePacksList, error)
 		PackInfo models.PackageInfo
 		CveIDs   []string
 	}
+
+	// { packageName: changelog-lines }
+	var rpm2changelog map[string]*string
+	allChangelog, err := o.getAllChangelog(packInfoList)
+	if err != nil {
+		o.log.Errorf("Failed to getAllchangelog. err: %s", err)
+		return nil, err
+	}
+	rpm2changelog, err = o.parseAllChangelog(allChangelog)
+	if err != nil {
+		return nil, fmt.Errorf("Failed to parseAllChangelog. err: %s", err)
+	}
+
 	var results []PackInfoCveIDs
 	for i, packInfo := range packInfoList {
-		changelog, err := o.getChangelog(packInfo.Name)
-		if err != nil {
-			o.log.Errorf("Failed to collect CVE IDs. err: %s", err)
-			return nil, err
-		}
+		changelog := o.getChangelogCVELines(rpm2changelog, packInfo)
 
 		// Collect unique set of CVE-ID in each changelog
 		uniqueCveIDMap := make(map[string]bool)
@@ -395,7 +369,6 @@ func (o *redhat) scanUnsecurePackagesUsingYumCheckUpdate() (CvePacksList, error)
 			//  CvssScore: cinfo.CvssScore(conf.Lang),
 		})
 	}
-	sort.Sort(CvePacksList(cvePacksList))
 	return cvePacksList, nil
 }
 
@@ -410,7 +383,9 @@ func (o *redhat) parseYumCheckUpdateLines(stdout string) (results models.Package
 			continue
 		}
 		if needToParse {
-			if strings.HasPrefix(line, "Obsoleting") {
+			if strings.HasPrefix(line, "Obsoleting") ||
+				strings.HasPrefix(line, "Security:") {
+				// see https://github.com/future-architect/vuls/issues/165
 				continue
 			}
 			candidate, err := o.parseYumCheckUpdateLine(line)
@@ -427,6 +402,7 @@ func (o *redhat) parseYumCheckUpdateLines(stdout string) (results models.Package
 			}
 			installed.NewVersion = candidate.NewVersion
 			installed.NewRelease = candidate.NewRelease
+			installed.Repository = candidate.Repository
 			results = append(results, installed)
 		}
 	}
@@ -435,7 +411,7 @@ func (o *redhat) parseYumCheckUpdateLines(stdout string) (results models.Package
 
 func (o *redhat) parseYumCheckUpdateLine(line string) (models.PackageInfo, error) {
 	fields := strings.Fields(line)
-	if len(fields) != 3 {
+	if len(fields) < 3 {
 		return models.PackageInfo{}, fmt.Errorf("Unknown format: %s", line)
 	}
 	splitted := strings.Split(fields[0], ".")
@@ -446,28 +422,146 @@ func (o *redhat) parseYumCheckUpdateLine(line string) (models.PackageInfo, error
 		packName = strings.Join(strings.Split(fields[0], ".")[0:(len(splitted)-1)], ".")
 	}
 
-	fields = strings.Split(fields[1], "-")
-	if len(fields) != 2 {
+	verfields := strings.Split(fields[1], "-")
+	if len(verfields) != 2 {
 		return models.PackageInfo{}, fmt.Errorf("Unknown format: %s", line)
 	}
-	version := fields[0]
-	release := fields[1]
+	version := o.regexpReplace(verfields[0], `^[0-9]+:`, "")
+	release := verfields[1]
+	repos := strings.Join(fields[2:len(fields)], " ")
+
 	return models.PackageInfo{
 		Name:       packName,
 		NewVersion: version,
 		NewRelease: release,
+		Repository: repos,
 	}, nil
 }
 
-func (o *redhat) getChangelog(packageNames string) (stdout string, err error) {
-	command := ""
-	if o.ServerInfo.User == "root" {
-		command = "yes no | "
+func (o *redhat) mkPstring() *string {
+	str := ""
+	return &str
+}
+
+func (o *redhat) regexpReplace(src string, pat string, rep string) string {
+	re := regexp.MustCompile(pat)
+	return re.ReplaceAllString(src, rep)
+}
+
+var changeLogCVEPattern = regexp.MustCompile(`CVE-[0-9]+-[0-9]+`)
+
+func (o *redhat) getChangelogCVELines(rpm2changelog map[string]*string, packInfo models.PackageInfo) string {
+	rpm := fmt.Sprintf("%s-%s-%s", packInfo.Name, packInfo.NewVersion, packInfo.NewRelease)
+	retLine := ""
+	if rpm2changelog[rpm] != nil {
+		lines := strings.Split(*rpm2changelog[rpm], "\n")
+		for _, line := range lines {
+			if changeLogCVEPattern.MatchString(line) {
+				retLine += fmt.Sprintf("%s\n", line)
+			}
+		}
 	}
+	return retLine
+}
+
+func (o *redhat) parseAllChangelog(allChangelog string) (map[string]*string, error) {
+	var majorVersion int
+	if 0 < len(o.Distro.Release) && o.Distro.Family == "centos" {
+		majorVersion, _ = strconv.Atoi(strings.Split(o.Distro.Release, ".")[0])
+	} else {
+		return nil, fmt.Errorf("Not implemented yet: %s", o.getDistro())
+	}
+
+	orglines := strings.Split(allChangelog, "\n")
+	tmpline := ""
+	var lines []string
+	var prev, now bool
+	var err error
+	for i := range orglines {
+		if majorVersion == 5 {
+			/* for CentOS5 (yum-util < 1.1.20) */
+			prev = false
+			now = false
+			if 0 < i {
+				prev, err = o.isRpmPackageNameLine(orglines[i-1])
+				if err != nil {
+					return nil, err
+				}
+			}
+			now, err = o.isRpmPackageNameLine(orglines[i])
+			if err != nil {
+				return nil, err
+			}
+			if prev && now {
+				tmpline = fmt.Sprintf("%s, %s", tmpline, orglines[i])
+				continue
+			}
+			if !prev && now {
+				tmpline = fmt.Sprintf("%s%s", tmpline, orglines[i])
+				continue
+			}
+			if tmpline != "" {
+				lines = append(lines, fmt.Sprintf("%s", tmpline))
+				tmpline = ""
+			}
+			lines = append(lines, fmt.Sprintf("%s", orglines[i]))
+		} else {
+			/* for CentOS6,7 (yum-util >= 1.1.20) */
+			line := orglines[i]
+			line = o.regexpReplace(line, `^ChangeLog for: `, "")
+			line = o.regexpReplace(line, `^\*\*\sNo\sChangeLog\sfor:.*`, "")
+			lines = append(lines, line)
+		}
+	}
+
+	rpm2changelog := make(map[string]*string)
+	writePointer := o.mkPstring()
+	for _, line := range lines {
+		match, err := o.isRpmPackageNameLine(line)
+		if err != nil {
+			return nil, err
+		}
+		if match {
+			rpms := strings.Split(line, ",")
+			pNewString := o.mkPstring()
+			writePointer = pNewString
+			for _, rpm := range rpms {
+				rpm = strings.TrimSpace(rpm)
+				rpm = o.regexpReplace(rpm, `^[0-9]+:`, "")
+				rpm = o.regexpReplace(rpm, `\.(i386|i486|i586|i686|k6|athlon|x86_64|noarch|ppc|alpha|sparc)$`, "")
+				rpm2changelog[rpm] = pNewString
+			}
+		} else {
+			if strings.HasPrefix(line, "Dependencies Resolved") {
+				return rpm2changelog, nil
+			}
+			*writePointer += fmt.Sprintf("%s\n", line)
+		}
+	}
+	return rpm2changelog, nil
+}
+
+// CentOS
+func (o *redhat) getAllChangelog(packInfoList models.PackageInfoList) (stdout string, err error) {
+	packageNames := ""
+	for _, packInfo := range packInfoList {
+		packageNames += fmt.Sprintf("%s ", packInfo.Name)
+	}
+
+	command := "echo N | "
 	if 0 < len(config.Conf.HTTPProxy) {
 		command += util.ProxyEnv()
 	}
-	command += fmt.Sprintf(" yum update --changelog %s | grep CVE", packageNames)
+
+	yumopts := ""
+	if o.getServerInfo().Enablerepo != "" {
+		yumopts = " --enablerepo=" + o.getServerInfo().Enablerepo
+	}
+	if config.Conf.SkipBroken {
+		yumopts += " --skip-broken"
+	}
+	// yum update --changelog doesn't have --color option.
+	command += fmt.Sprintf(" LANGUAGE=en_US.UTF-8 yum %s --changelog update ", yumopts) + packageNames
 
 	r := o.ssh(command, sudo)
 	if !r.isSuccess(0, 1) {
@@ -484,42 +578,36 @@ type distroAdvisoryCveIDs struct {
 }
 
 // Scaning unsecure packages using yum-plugin-security.
-//TODO return whether already expired.
+// Amazon, RHEL
 func (o *redhat) scanUnsecurePackagesUsingYumPluginSecurity() (CvePacksList, error) {
-	if o.Family == "centos" {
+	if o.Distro.Family == "centos" {
 		// CentOS has no security channel.
 		// So use yum check-update && parse changelog
 		return CvePacksList{}, fmt.Errorf(
 			"yum updateinfo is not suppported on CentOS")
 	}
 
-	cmd := "yum repolist"
-	r := o.ssh(util.PrependProxyEnv(cmd), sudo)
+	cmd := "yum --color=never repolist"
+	r := o.ssh(util.PrependProxyEnv(cmd), o.sudo())
 	if !r.isSuccess() {
-		return nil, fmt.Errorf(
-			"Failed to %s. status: %d, stdout: %s, stderr: %s",
-			cmd, r.ExitStatus, r.Stdout, r.Stderr)
+		return nil, fmt.Errorf("Failed to SSH: %s", r)
 	}
 
 	// get advisoryID(RHSA, ALAS) - package name,version
-	cmd = "yum updateinfo list available --security"
-	r = o.ssh(util.PrependProxyEnv(cmd), sudo)
+	cmd = "yum --color=never updateinfo list available --security"
+	r = o.ssh(util.PrependProxyEnv(cmd), o.sudo())
 	if !r.isSuccess() {
-		return nil, fmt.Errorf(
-			"Failed to %s. status: %d, stdout: %s, stderr: %s",
-			cmd, r.ExitStatus, r.Stdout, r.Stderr)
+		return nil, fmt.Errorf("Failed to SSH: %s", r)
 	}
 	advIDPackNamesList, err := o.parseYumUpdateinfoListAvailable(r.Stdout)
 
 	// get package name, version, rel to be upgrade.
 	//  cmd = "yum check-update --security"
-	cmd = "yum check-update"
-	r = o.ssh(util.PrependProxyEnv(cmd), sudo)
+	cmd = "LANGUAGE=en_US.UTF-8 yum --color=never check-update"
+	r = o.ssh(util.PrependProxyEnv(cmd), o.sudo())
 	if !r.isSuccess(0, 100) {
 		//returns an exit code of 100 if there are available updates.
-		return nil, fmt.Errorf(
-			"Failed to %s. status: %d, stdout: %s, stderr: %s",
-			cmd, r.ExitStatus, r.Stdout, r.Stderr)
+		return nil, fmt.Errorf("Failed to SSH: %s", r)
 	}
 	updatable, err := o.parseYumCheckUpdateLines(r.Stdout)
 	if err != nil {
@@ -543,12 +631,10 @@ func (o *redhat) scanUnsecurePackagesUsingYumPluginSecurity() (CvePacksList, err
 	}
 
 	// get advisoryID(RHSA, ALAS) - CVE IDs
-	cmd = "yum updateinfo --security update"
-	r = o.ssh(util.PrependProxyEnv(cmd), sudo)
+	cmd = "yum --color=never updateinfo --security update"
+	r = o.ssh(util.PrependProxyEnv(cmd), o.sudo())
 	if !r.isSuccess() {
-		return nil, fmt.Errorf(
-			"Failed to %s. status: %d, stdout: %s, stderr: %s",
-			cmd, r.ExitStatus, r.Stdout, r.Stderr)
+		return nil, fmt.Errorf("Failed to SSH: %s", r)
 	}
 	advisoryCveIDsList, err := o.parseYumUpdateinfo(r.Stdout)
 	if err != nil {
@@ -596,6 +682,8 @@ func (o *redhat) scanUnsecurePackagesUsingYumPluginSecurity() (CvePacksList, err
 	return result, nil
 }
 
+var horizontalRulePattern = regexp.MustCompile(`^=+$`)
+
 func (o *redhat) parseYumUpdateinfo(stdout string) (result []distroAdvisoryCveIDs, err error) {
 	sectionState := Outside
 	lines := strings.Split(stdout, "\n")
@@ -613,7 +701,7 @@ func (o *redhat) parseYumUpdateinfo(stdout string) (result []distroAdvisoryCveID
 		line = strings.TrimSpace(line)
 
 		// find the new section pattern
-		if match, _ := o.isHorizontalRule(line); match {
+		if horizontalRulePattern.MatchString(line) {
 
 			// set previous section's result to return-variable
 			if sectionState == Content {
@@ -640,7 +728,7 @@ func (o *redhat) parseYumUpdateinfo(stdout string) (result []distroAdvisoryCveID
 
 		switch sectionState {
 		case Header:
-			switch o.Family {
+			switch o.Distro.Family {
 			case "centos":
 				// CentOS has no security channel.
 				// So use yum check-update && parse changelog
@@ -707,8 +795,22 @@ func (o *redhat) changeSectionState(state int) (newState int) {
 	return newState
 }
 
-func (o *redhat) isHorizontalRule(line string) (bool, error) {
-	return regexp.MatchString("^=+$", line)
+var rpmPackageArchPattern = regexp.MustCompile(
+	`^[^ ]+\.(i386|i486|i586|i686|k6|athlon|x86_64|noarch|ppc|alpha|sparc)$`)
+
+func (o *redhat) isRpmPackageNameLine(line string) (bool, error) {
+	s := strings.TrimPrefix(line, "ChangeLog for: ")
+	ss := strings.Split(s, ", ")
+	if len(ss) == 0 {
+		return false, nil
+	}
+	for _, s := range ss {
+		s = strings.TrimRight(s, " \r\n")
+		if !rpmPackageArchPattern.MatchString(s) {
+			return false, nil
+		}
+	}
+	return true, nil
 }
 
 // see test case
@@ -728,9 +830,10 @@ func (o *redhat) parseYumUpdateinfoHeaderCentOS(line string) (packs []models.Pac
 	return
 }
 
+var yumHeaderPattern = regexp.MustCompile(`(ALAS-.+): (.+) priority package update for (.+)$`)
+
 func (o *redhat) parseYumUpdateinfoHeaderAmazon(line string) (a models.DistroAdvisory, names []string, err error) {
-	re, _ := regexp.Compile(`(ALAS-.+): (.+) priority package update for (.+)$`)
-	result := re.FindStringSubmatch(line)
+	result := yumHeaderPattern.FindStringSubmatch(line)
 	if len(result) == 4 {
 		a.AdvisoryID = result[1]
 		a.Severity = result[2]
@@ -742,31 +845,36 @@ func (o *redhat) parseYumUpdateinfoHeaderAmazon(line string) (a models.DistroAdv
 	return
 }
 
+var yumCveIDPattern = regexp.MustCompile(`(CVE-\d{4}-\d{4,})`)
+
 func (o *redhat) parseYumUpdateinfoLineToGetCveIDs(line string) []string {
-	re, _ := regexp.Compile(`(CVE-\d{4}-\d{4})`)
-	return re.FindAllString(line, -1)
+	return yumCveIDPattern.FindAllString(line, -1)
 }
 
+var yumAdvisoryIDPattern = regexp.MustCompile(`^ *Update ID : (.*)$`)
+
 func (o *redhat) parseYumUpdateinfoToGetAdvisoryID(line string) (advisoryID string, found bool) {
-	re, _ := regexp.Compile(`^ *Update ID : (.*)$`)
-	result := re.FindStringSubmatch(line)
+	result := yumAdvisoryIDPattern.FindStringSubmatch(line)
 	if len(result) != 2 {
 		return "", false
 	}
 	return strings.TrimSpace(result[1]), true
 }
 
+var yumIssuedPattern = regexp.MustCompile(`^\s*Issued : (\d{4}-\d{2}-\d{2})`)
+
 func (o *redhat) parseYumUpdateinfoLineToGetIssued(line string) (date time.Time, found bool) {
-	return o.parseYumUpdateinfoLineToGetDate(line, `^\s*Issued : (\d{4}-\d{2}-\d{2})`)
+	return o.parseYumUpdateinfoLineToGetDate(line, yumIssuedPattern)
 }
 
+var yumUpdatedPattern = regexp.MustCompile(`^\s*Updated : (\d{4}-\d{2}-\d{2})`)
+
 func (o *redhat) parseYumUpdateinfoLineToGetUpdated(line string) (date time.Time, found bool) {
-	return o.parseYumUpdateinfoLineToGetDate(line, `^\s*Updated : (\d{4}-\d{2}-\d{2})`)
+	return o.parseYumUpdateinfoLineToGetDate(line, yumUpdatedPattern)
 }
 
-func (o *redhat) parseYumUpdateinfoLineToGetDate(line, regexpFormat string) (date time.Time, found bool) {
-	re, _ := regexp.Compile(regexpFormat)
-	result := re.FindStringSubmatch(line)
+func (o *redhat) parseYumUpdateinfoLineToGetDate(line string, regexpPattern *regexp.Regexp) (date time.Time, found bool) {
+	result := regexpPattern.FindStringSubmatch(line)
 	if len(result) != 2 {
 		return date, false
 	}
@@ -777,14 +885,16 @@ func (o *redhat) parseYumUpdateinfoLineToGetDate(line, regexpFormat string) (dat
 	return t, true
 }
 
+var yumDescriptionPattern = regexp.MustCompile(`^\s*Description : `)
+
 func (o *redhat) isDescriptionLine(line string) bool {
-	re, _ := regexp.Compile(`^\s*Description : `)
-	return re.MatchString(line)
+	return yumDescriptionPattern.MatchString(line)
 }
 
+var yumSeverityPattern = regexp.MustCompile(`^ *Severity : (.*)$`)
+
 func (o *redhat) parseYumUpdateinfoToGetSeverity(line string) (severity string, found bool) {
-	re, _ := regexp.Compile(`^ *Severity : (.*)$`)
-	result := re.FindStringSubmatch(line)
+	result := yumSeverityPattern.FindStringSubmatch(line)
 	if len(result) != 2 {
 		return "", false
 	}
@@ -859,3 +969,16 @@ func (o *redhat) parseYumUpdateinfoListAvailable(stdout string) (advisoryIDPacks
 	}
 	return result, nil
 }
+
+func (o *redhat) clone() osTypeInterface {
+	return o
+}
+
+func (o *redhat) sudo() bool {
+	switch o.Distro.Family {
+	case "amazon":
+		return false
+	default:
+		return true
+	}
+}

scan/redhat_test.go

@@ -143,11 +143,11 @@ func TestParseYumUpdateinfoLineToGetCveIDs(t *testing.T) {
 			[]string{"CVE-2015-0278"},
 		},
 		{
-			": 1195457 - nodejs-0.10.35 causes undefined symbolsCVE-2015-0278, CVE-2015-0278, CVE-2015-0277",
+			": 1195457 - nodejs-0.10.35 causes undefined symbolsCVE-2015-0278, CVE-2015-0278, CVE-2015-02770000000 ",
 			[]string{
 				"CVE-2015-0278",
 				"CVE-2015-0278",
-				"CVE-2015-0277",
+				"CVE-2015-02770000000",
 			},
 		},
 	}
@@ -302,6 +302,54 @@ func TestIsDescriptionLine(t *testing.T) {
 	}
 }
 
+func TestIsRpmPackageNameLine(t *testing.T) {
+	r := newRedhat(config.ServerInfo{})
+	var tests = []struct {
+		in    string
+		found bool
+	}{
+		{
+			"stunnel-4.15-2.el5.2.i386",
+			true,
+		},
+		{
+			"iproute-2.6.18-15.el5.i386",
+			true,
+		},
+		{
+			"1:yum-updatesd-0.9-6.el5_10.noarch",
+			true,
+		},
+		{
+			"glibc-2.12-1.192.el6.x86_64",
+			true,
+		},
+		{
+			" glibc-2.12-1.192.el6.x86_64",
+			false,
+		},
+		{
+			"glibc-2.12-1.192.el6.x86_64, iproute-2.6.18-15.el5.i386",
+			true,
+		},
+		{
+			"k6 hoge.i386",
+			false,
+		},
+		{
+			"triathlon",
+			false,
+		},
+	}
+
+	for i, tt := range tests {
+		found, err := r.isRpmPackageNameLine(tt.in)
+		if tt.found != found {
+			t.Errorf("[%d] line: %s, expected %t, actual %t, err %v", i, tt.in, tt.found, found, err)
+		}
+	}
+}
+
 func TestParseYumUpdateinfoToGetSeverity(t *testing.T) {
 	r := newRedhat(config.ServerInfo{})
 	var tests = []struct {
@@ -403,7 +451,7 @@ Description : The Berkeley Internet Name Domain (BIND) is an implementation of
 	updated, _ := time.Parse("2006-01-02", "2015-09-04")
 
 	r := newRedhat(config.ServerInfo{})
-	r.Family = "redhat"
+	r.Distro = config.Distro{Family: "redhat"}
 
 	var tests = []struct {
 		in  string
@@ -463,7 +511,7 @@ Description : The Berkeley Internet Name Domain (BIND) is an implementation of
 func TestParseYumUpdateinfoAmazon(t *testing.T) {
 
 	r := newRedhat(config.ServerInfo{})
-	r.Family = "amazon"
+	r.Distro = config.Distro{Family: "redhat"}
 
 	issued, _ := time.Parse("2006-01-02", "2015-12-15")
 	updated, _ := time.Parse("2006-01-02", "2015-12-16")
@@ -553,7 +601,7 @@ Description : Package updates are available for Amazon Linux AMI that fix the
 
 func TestParseYumCheckUpdateLines(t *testing.T) {
 	r := newRedhat(config.ServerInfo{})
-	r.Family = "centos"
+	r.Distro = config.Distro{Family: "centos"}
 	stdout := `Loaded plugins: changelog, fastestmirror, keys, protect-packages, protectbase, security
 Loading mirror speeds from cached hostfile
  * base: mirror.fairway.ne.jp
@@ -567,6 +615,8 @@ bash.x86_64                    4.1.2-33.el6_7.1              updates
 Obsoleting Packages
 python-libs.i686    2.6.6-64.el6   rhui-REGION-rhel-server-releases
     python-ordereddict.noarch     1.1-3.el6ev    installed
+bind-utils.x86_64                       30:9.3.6-25.P1.el5_11.8          updates
+pytalloc.x86_64                 2.0.7-2.el6                      @CentOS 6.5/6.5
 `
 
 	r.Packages = []models.PackageInfo{
@@ -590,6 +640,16 @@ python-libs.i686    2.6.6-64.el6   rhui-REGION-rhel-server-releases
 			Version: "1.0",
 			Release: "1",
 		},
+		{
+			Name:    "bind-utils",
+			Version: "1.0",
+			Release: "1",
+		},
+		{
+			Name:    "pytalloc",
+			Version: "2.0.1",
+			Release: "0",
+		},
 	}
 	var tests = []struct {
 		in  string
@@ -604,6 +664,7 @@ python-libs.i686    2.6.6-64.el6   rhui-REGION-rhel-server-releases
 					Release:    "4.el6",
 					NewVersion: "2.3.7",
 					NewRelease: "5.el6",
+					Repository: "base",
 				},
 				{
 					Name:       "bash",
@@ -611,6 +672,7 @@ python-libs.i686    2.6.6-64.el6   rhui-REGION-rhel-server-releases
 					Release:    "33",
 					NewVersion: "4.1.2",
 					NewRelease: "33.el6_7.1",
+					Repository: "updates",
 				},
 				{
 					Name:       "python-libs",
@@ -618,6 +680,7 @@ python-libs.i686    2.6.6-64.el6   rhui-REGION-rhel-server-releases
 					Release:    "1.1-0",
 					NewVersion: "2.6.6",
 					NewRelease: "64.el6",
+					Repository: "rhui-REGION-rhel-server-releases",
 				},
 				{
 					Name:       "python-ordereddict",
@@ -625,6 +688,23 @@ python-libs.i686    2.6.6-64.el6   rhui-REGION-rhel-server-releases
 					Release:    "1",
 					NewVersion: "1.1",
 					NewRelease: "3.el6ev",
+					Repository: "installed",
+				},
+				{
+					Name:       "bind-utils",
+					Version:    "1.0",
+					Release:    "1",
+					NewVersion: "9.3.6",
+					NewRelease: "25.P1.el5_11.8",
+					Repository: "updates",
+				},
+				{
+					Name:       "pytalloc",
+					Version:    "2.0.1",
+					Release:    "0",
+					NewVersion: "2.0.7",
+					NewRelease: "2.el6",
+					Repository: "@CentOS 6.5/6.5",
 				},
 			},
 		},
@@ -648,23 +728,23 @@ python-libs.i686    2.6.6-64.el6   rhui-REGION-rhel-server-releases
 
 func TestParseYumCheckUpdateLinesAmazon(t *testing.T) {
 	r := newRedhat(config.ServerInfo{})
-	r.Family = "amzon"
+	r.Distro = config.Distro{Family: "amazon"}
 	stdout := `Loaded plugins: priorities, update-motd, upgrade-helper
 34 package(s) needed for security, out of 71 available
 
 bind-libs.x86_64           32:9.8.2-0.37.rc1.45.amzn1      amzn-main
-java-1.7.0-openjdk.x86_64  1:1.7.0.95-2.6.4.0.65.amzn1     amzn-main
+java-1.7.0-openjdk.x86_64  1.7.0.95-2.6.4.0.65.amzn1     amzn-main
 if-not-architecture        100-200                         amzn-main
 `
 	r.Packages = []models.PackageInfo{
 		{
 			Name:    "bind-libs",
-			Version: "32:9.8.0",
+			Version: "9.8.0",
 			Release: "0.33.rc1.45.amzn1",
 		},
 		{
 			Name:    "java-1.7.0-openjdk",
-			Version: "1:1.7.0.0",
+			Version: "1.7.0.0",
 			Release: "2.6.4.0.0.amzn1",
 		},
 		{
@@ -682,17 +762,19 @@ if-not-architecture        100-200                         amzn-main
 			models.PackageInfoList{
 				{
 					Name:       "bind-libs",
-					Version:    "32:9.8.0",
+					Version:    "9.8.0",
 					Release:    "0.33.rc1.45.amzn1",
-					NewVersion: "32:9.8.2",
+					NewVersion: "9.8.2",
 					NewRelease: "0.37.rc1.45.amzn1",
+					Repository: "amzn-main",
 				},
 				{
 					Name:       "java-1.7.0-openjdk",
-					Version:    "1:1.7.0.0",
+					Version:    "1.7.0.0",
 					Release:    "2.6.4.0.0.amzn1",
-					NewVersion: "1:1.7.0.95",
+					NewVersion: "1.7.0.95",
 					NewRelease: "2.6.4.0.65.amzn1",
+					Repository: "amzn-main",
 				},
 				{
 					Name:       "if-not-architecture",
@@ -700,6 +782,7 @@ if-not-architecture        100-200                         amzn-main
 					Release:    "20",
 					NewVersion: "100",
 					NewRelease: "200",
+					Repository: "amzn-main",
 				},
 			},
 		},
@@ -844,3 +927,309 @@ func TestExtractPackNameVerRel(t *testing.T) {
 	}
 
 }
+
+const (
+	/* for CentOS6,7 (yum-util >= 1.1.20) */
+	stdoutCentos6 = `---> Package libaio.x86_64 0:0.3.107-10.el6 will be installed
+--> Finished Dependency Resolution
+
+Changes in packages about to be updated:
+
+ChangeLog for: binutils-2.20.51.0.2-5.44.el6.x86_64
+* Mon Dec  7 21:00:00 2015 Nick Clifton <nickc@redhat.com> - 2.20.51.0.2-5.44
+- Backport upstream RELRO fixes. (#1227839)
+
+** No ChangeLog for: chkconfig-1.3.49.5-1.el6.x86_64
+
+ChangeLog for: coreutils-8.4-43.el6.x86_64, coreutils-libs-8.4-43.el6.x86_64
+* Wed Feb 10 21:00:00 2016 Ondrej Vasik <ovasik@redhat.com> - 8.4-43
+- sed should actually be /bin/sed (related #1222140)
+
+* Wed Jan  6 21:00:00 2016 Ondrej Vasik <ovasik@redhat.com> - 8.4-41
+- colorls.sh,colorls.csh - call utilities with complete path (#1222140)
+- mkdir, mkfifo, mknod - respect default umask/acls when
+  COREUTILS_CHILD_DEFAULT_ACLS envvar is set (to match rhel 7 behaviour,
+
+ChangeLog for: centos-release-6-8.el6.centos.12.3.x86_64
+* Wed May 18 21:00:00 2016 Johnny Hughes <johnny@centos.org> 6-8.el6.centos.12.3
+- CentOS-6.8 Released
+- TESTSTRING CVE-0000-0000
+
+ChangeLog for: 12:dhclient-4.1.1-51.P1.el6.centos.x86_64, 12:dhcp-common-4.1.1-51.P1.el6.centos.x86_64
+* Tue May 10 21:00:00 2016 Johnny Hughes <johnny@centos.org> - 12:4.1.1-51.P1
+- created patch 1000 for CentOS Branding
+- replaced vvendor variable with CentOS in the SPEC file
+- TESTSTRING CVE-1111-1111
+
+* Mon Jan 11 21:00:00 2016 Jiri Popelka <jpopelka@redhat.com> - 12:4.1.1-51.P1
+- send unicast request/release via correct interface (#1297445)
+
+* Thu Dec  3 21:00:00 2015 Jiri Popelka <jpopelka@redhat.com> - 12:4.1.1-50.P1
+- Lease table overflow crash. (#1133917)
+- Add ignore-client-uids option. (#1196768)
+- dhclient-script: it's OK if the arping reply comes from our system. (#1204095)
+- VLAN ID is only bottom 12-bits of TCI. (#1259552)
+- dhclient: Make sure link-local address is ready in stateless mode. (#1263466)
+- dhclient-script: make_resolv_conf(): Keep old nameservers
+  if server sends domain-name/search, but no nameservers. (#1269595)
+
+ChangeLog for: file-5.04-30.el6.x86_64, file-libs-5.04-30.el6.x86_64
+* Tue Feb 16 21:00:00 2016 Jan Kaluza <jkaluza@redhat.com> 5.04-30
+- fix CVE-2014-3538 (unrestricted regular expression matching)
+
+* Tue Jan  5 21:00:00 2016 Jan Kaluza <jkaluza@redhat.com> 5.04-29
+- fix #1284826 - try to read ELF header to detect corrupted one
+
+* Wed Dec 16 21:00:00 2015 Jan Kaluza <jkaluza@redhat.com> 5.04-28
+- fix #1263987 - fix bugs found by coverity in the patch
+
+* Thu Nov 26 21:00:00 2015 Jan Kaluza <jkaluza@redhat.com> 5.04-27
+- fix CVE-2014-3587 (incomplete fix for CVE-2012-1571)
+- fix CVE-2014-3710 (out-of-bounds read in elf note headers)
+- fix CVE-2014-8116 (multiple DoS issues (resource consumption))
+- fix CVE-2014-8117 (denial of service issue (resource consumption))
+- fix CVE-2014-9620 (limit the number of ELF notes processed)
+- fix CVE-2014-9653 (malformed elf file causes access to uninitialized memory)
+
+
+Dependencies Resolved
+
+`
+	/* for CentOS5 (yum-util < 1.1.20) */
+	stdoutCentos5 = `---> Package portmap.i386 0:4.0-65.2.2.1 set to be updated
+--> Finished Dependency Resolution
+
+Changes in packages about to be updated:
+
+libuser-0.54.7-3.el5.i386
+nss_db-2.2-38.el5_11.i386
+* Thu Nov 20 23:00:00 2014 Nalin Dahyabhai <nalin@redhat.com> - 2.2-38
+- build without strict aliasing (internal build tooling)
+
+* Sat Nov 15 23:00:00 2014 Nalin Dahyabhai <nalin@redhat.com> - 2.2-37
+- pull in fix for a memory leak in nss_db (#1163493)
+
+acpid-1.0.4-12.el5.i386
+* Thu Oct  6 00:00:00 2011 Jiri Skala <jskala@redhat.com> - 1.0.4-12
+- Resolves: #729769 - acpid dumping useless info to log
+
+nash-5.1.19.6-82.el5.i386, mkinitrd-5.1.19.6-82.el5.i386
+* Tue Apr 15 00:00:00 2014 Brian C. Lane <bcl@redhat.com> 5.1.19.6-82
+- Use ! instead of / when searching sysfs for ccis device
+  Resolves: rhbz#988020
+- Always include ahci module (except on s390) (bcl)
+  Resolves: rhbz#978245
+- Prompt to recreate default initrd (karsten)
+  Resolves: rhbz#472764
+
+util-linux-2.13-0.59.el5_8.i386
+* Wed Oct 17 00:00:00 2012 Karel Zak <kzak@redhat.com> 2.13-0.59.el5_8
+- fix #865791 - fdisk fails to partition disk not in use
+
+* Wed Dec 21 23:00:00 2011 Karel Zak <kzak@redhat.com> 2.13-0.59
+- fix #768382 - CVE-2011-1675 CVE-2011-1677 util-linux various flaws
+
+* Wed Oct 26 00:00:00 2011 Karel Zak <kzak@redhat.com> 2.13-0.58
+- fix #677452 - util-linux fails to build with gettext-0.17
+
+30:bind-utils-9.3.6-25.P1.el5_11.8.i386, 30:bind-libs-9.3.6-25.P1.el5_11.8.i386
+* Mon Mar 14 23:00:00 2016 Tomas Hozza <thozza@redhat.com> - 30:9.3.6-25.P1.8
+- Fix issue with patch for CVE-2016-1285 and CVE-2016-1286 found by test suite
+
+* Wed Mar  9 23:00:00 2016 Tomas Hozza <thozza@redhat.com> - 30:9.3.6-25.P1.7
+- Fix CVE-2016-1285 and CVE-2016-1286
+
+* Mon Jan 18 23:00:00 2016 Tomas Hozza <thozza@redhat.com> - 30:9.3.6-25.P1.6
+- Fix CVE-2015-8704
+
+* Thu Sep  3 00:00:00 2015 Tomas Hozza <thozza@redhat.com> - 30:9.3.6-25.P1.5
+- Fix CVE-2015-8000
+
+
+Dependencies Resolved
+
+`
+)
+
+func TestGetChangelogCVELines(t *testing.T) {
+	var testsCentos6 = []struct {
+		in  models.PackageInfo
+		out string
+	}{
+		{
+			models.PackageInfo{
+				Name:       "binutils",
+				NewVersion: "2.20.51.0.2",
+				NewRelease: "5.44.el6",
+			},
+			"",
+		},
+		{
+			models.PackageInfo{
+				Name:       "centos-release",
+				NewVersion: "6",
+				NewRelease: "8.el6.centos.12.3",
+			},
+			`- TESTSTRING CVE-0000-0000
+`,
+		},
+		{
+			models.PackageInfo{
+				Name:       "dhclient",
+				NewVersion: "4.1.1",
+				NewRelease: "51.P1.el6.centos",
+			},
+			`- TESTSTRING CVE-1111-1111
+`,
+		},
+		{
+			models.PackageInfo{
+				Name:       "dhcp-common",
+				NewVersion: "4.1.1",
+				NewRelease: "51.P1.el6.centos",
+			},
+			`- TESTSTRING CVE-1111-1111
+`,
+		},
+		{
+			models.PackageInfo{
+				Name:       "coreutils-libs",
+				NewVersion: "8.4",
+				NewRelease: "43.el6",
+			},
+			"",
+		},
+		{
+			models.PackageInfo{
+				Name:       "file",
+				NewVersion: "5.04",
+				NewRelease: "30.el6",
+			},
+			`- fix CVE-2014-3538 (unrestricted regular expression matching)
+- fix CVE-2014-3587 (incomplete fix for CVE-2012-1571)
+- fix CVE-2014-3710 (out-of-bounds read in elf note headers)
+- fix CVE-2014-8116 (multiple DoS issues (resource consumption))
+- fix CVE-2014-8117 (denial of service issue (resource consumption))
+- fix CVE-2014-9620 (limit the number of ELF notes processed)
+- fix CVE-2014-9653 (malformed elf file causes access to uninitialized memory)
+`,
+		},
+		{
+			models.PackageInfo{
+				Name:       "file-libs",
+				NewVersion: "5.04",
+				NewRelease: "30.el6",
+			},
+			`- fix CVE-2014-3538 (unrestricted regular expression matching)
+- fix CVE-2014-3587 (incomplete fix for CVE-2012-1571)
+- fix CVE-2014-3710 (out-of-bounds read in elf note headers)
+- fix CVE-2014-8116 (multiple DoS issues (resource consumption))
+- fix CVE-2014-8117 (denial of service issue (resource consumption))
+- fix CVE-2014-9620 (limit the number of ELF notes processed)
+- fix CVE-2014-9653 (malformed elf file causes access to uninitialized memory)
+`,
+		},
+	}
+
+	r := newRedhat(config.ServerInfo{})
+	r.Distro = config.Distro{
+		Family:  "centos",
+		Release: "6.7",
+	}
+	for _, tt := range testsCentos6 {
+		rpm2changelog, err := r.parseAllChangelog(stdoutCentos6)
+		if err != nil {
+			t.Errorf("err: %s", err)
+		}
+		changelog := r.getChangelogCVELines(rpm2changelog, tt.in)
+		if tt.out != changelog {
+			t.Errorf("line: expected %s, actual %s, tt: %#v", tt.out, changelog, tt)
+		}
+	}
+
+	var testsCentos5 = []struct {
+		in  models.PackageInfo
+		out string
+	}{
+		{
+			models.PackageInfo{
+				Name:       "libuser",
+				NewVersion: "0.54.7",
+				NewRelease: "3.el5",
+			},
+			"",
+		},
+		{
+			models.PackageInfo{
+				Name:       "nss_db",
+				NewVersion: "2.2",
+				NewRelease: "38.el5_11",
+			},
+			"",
+		},
+		{
+			models.PackageInfo{
+				Name:       "acpid",
+				NewVersion: "1.0.4",
+				NewRelease: "82.el5",
+			},
+			"",
+		},
+		{
+			models.PackageInfo{
+				Name:       "mkinitrd",
+				NewVersion: "5.1.19.6",
+				NewRelease: "82.el5",
+			},
+			"",
+		},
+		{
+			models.PackageInfo{
+				Name:       "util-linux",
+				NewVersion: "2.13",
+				NewRelease: "0.59.el5_8",
+			},
+			`- fix #768382 - CVE-2011-1675 CVE-2011-1677 util-linux various flaws
+`,
+		},
+		{
+			models.PackageInfo{
+				Name:       "bind-libs",
+				NewVersion: "9.3.6",
+				NewRelease: "25.P1.el5_11.8",
+			},
+			`- Fix issue with patch for CVE-2016-1285 and CVE-2016-1286 found by test suite
+- Fix CVE-2016-1285 and CVE-2016-1286
+- Fix CVE-2015-8704
+- Fix CVE-2015-8000
+`,
+		},
+		{
+			models.PackageInfo{
+				Name:       "bind-utils",
+				NewVersion: "9.3.6",
+				NewRelease: "25.P1.el5_11.8",
+			},
+			`- Fix issue with patch for CVE-2016-1285 and CVE-2016-1286 found by test suite
+- Fix CVE-2016-1285 and CVE-2016-1286
+- Fix CVE-2015-8704
+- Fix CVE-2015-8000
+`,
+		},
+	}
+
+	r.Distro = config.Distro{
+		Family:  "centos",
+		Release: "5.6",
+	}
+	for _, tt := range testsCentos5 {
+		rpm2changelog, err := r.parseAllChangelog(stdoutCentos5)
+		if err != nil {
+			t.Errorf("err: %s", err)
+		}
+		changelog := r.getChangelogCVELines(rpm2changelog, tt.in)
+		if tt.out != changelog {
+			t.Errorf("line: expected %s, actual %s, tt: %#v", tt.out, changelog, tt)
+		}
+	}
+}

scan/serverapi.go

@@ -1,10 +1,31 @@
+/* Vuls - Vulnerability Scanner
+Copyright (C) 2016  Future Architect, Inc. Japan.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
 package scan
 
 import (
+	"bufio"
 	"fmt"
+	"os"
+	"strings"
 	"time"
 
 	"github.com/Sirupsen/logrus"
+	"github.com/future-architect/vuls/cache"
 	"github.com/future-architect/vuls/config"
 	"github.com/future-architect/vuls/models"
 	cve "github.com/kotakanbe/go-cve-dictionary/models"
@@ -15,20 +36,37 @@ var Log *logrus.Entry
 
 var servers []osTypeInterface
 
-// Base Interface of redhat, debian
+// Base Interface of redhat, debian, freebsd
 type osTypeInterface interface {
 	setServerInfo(config.ServerInfo)
 	getServerInfo() config.ServerInfo
-	setDistributionInfo(string, string)
-	getDistributionInfo() string
+
+	setDistro(string, string)
+	getDistro() config.Distro
+
+	// checkDependencies checks if dependencies are installed on the target server.
+	checkDependencies() error
+	getLackDependencies() []string
+
+	checkIfSudoNoPasswd() error
+	detectPlatform() error
+	getPlatform() models.Platform
+
 	checkRequiredPackagesInstalled() error
 	scanPackages() error
 	scanVulnByCpeName() error
 	install() error
 	convertToModel() (models.ScanResult, error)
+
+	runningContainers() ([]config.Container, error)
+	exitedContainers() ([]config.Container, error)
+	allContainers() ([]config.Container, error)
+
+	getErrs() []error
+	setErrs([]error)
 }
 
-// osPackages included by linux struct
+// osPackages is included by base struct
 type osPackages struct {
 	// installed packages
 	Packages models.PackageInfoList
@@ -52,10 +90,9 @@ type CvePacksList []CvePacksInfo
 type CvePacksInfo struct {
 	CveID            string
 	CveDetail        cve.CveDetail
-	Packs            []models.PackageInfo
-	DistroAdvisories []models.DistroAdvisory // for Aamazon, RHEL
+	Packs            models.PackageInfoList
+	DistroAdvisories []models.DistroAdvisory // for Aamazon, RHEL, FreeBSD
 	CpeNames         []string
-	//  CvssScore float64
 }
 
 // FindByCveID find by CVEID
@@ -91,75 +128,368 @@ func (s CvePacksList) Swap(i, j int) {
 
 // Less implement Sort Interface
 func (s CvePacksList) Less(i, j int) bool {
-	return s[i].CveDetail.CvssScore("en") > s[j].CveDetail.CvssScore("en")
+	return s[i].CveDetail.CvssScore(config.Conf.Lang) >
+		s[j].CveDetail.CvssScore(config.Conf.Lang)
 }
 
-func detectOs(c config.ServerInfo) (osType osTypeInterface) {
+func detectOS(c config.ServerInfo) (osType osTypeInterface) {
 	var itsMe bool
-	itsMe, osType = detectDebian(c)
-	if itsMe {
+	var fatalErr error
+
+	itsMe, osType, fatalErr = detectDebian(c)
+	if fatalErr != nil {
+		osType.setServerInfo(c)
+		osType.setErrs([]error{fatalErr})
+		return
+	} else if itsMe {
+		Log.Debugf("Debian like Linux. Host: %s:%s", c.Host, c.Port)
 		return
 	}
-	itsMe, osType = detectRedhat(c)
+
+	if itsMe, osType = detectRedhat(c); itsMe {
+		Log.Debugf("Redhat like Linux. Host: %s:%s", c.Host, c.Port)
+		return
+	}
+	if itsMe, osType = detectFreebsd(c); itsMe {
+		Log.Debugf("FreeBSD. Host: %s:%s", c.Host, c.Port)
+		return
+	}
+	osType.setServerInfo(c)
+	osType.setErrs([]error{fmt.Errorf("Unknown OS Type")})
 	return
 }
 
+// PrintSSHableServerNames print SSH-able servernames
+func PrintSSHableServerNames() {
+	Log.Info("SSH-able servers are below...")
+	for _, s := range servers {
+		if s.getServerInfo().IsContainer() {
+			fmt.Printf("%s@%s ",
+				s.getServerInfo().Container.Name,
+				s.getServerInfo().ServerName,
+			)
+		} else {
+			fmt.Printf("%s ", s.getServerInfo().ServerName)
+		}
+	}
+	fmt.Printf("\n")
+}
+
 // InitServers detect the kind of OS distribution of target servers
-func InitServers(localLogger *logrus.Entry) (err error) {
+func InitServers(localLogger *logrus.Entry) error {
 	Log = localLogger
-	if servers, err = detectServersOS(); err != nil {
-		err = fmt.Errorf("Failed to detect the type of OS. err: %s", err)
+	servers = detectServerOSes()
+	if len(servers) == 0 {
+		return fmt.Errorf("No scannable servers")
 	}
-	return
+
+	containers := detectContainerOSes()
+	if config.Conf.ContainersOnly {
+		servers = containers
+	} else {
+		servers = append(servers, containers...)
+	}
+	return nil
 }
 
-func detectServersOS() (osi []osTypeInterface, err error) {
+func detectServerOSes() (sshAbleOses []osTypeInterface) {
+	Log.Info("Detecting OS of servers... ")
 	osTypeChan := make(chan osTypeInterface, len(config.Conf.Servers))
 	defer close(osTypeChan)
 	for _, s := range config.Conf.Servers {
 		go func(s config.ServerInfo) {
-			osTypeChan <- detectOs(s)
+			defer func() {
+				if p := recover(); p != nil {
+					Log.Debugf("Panic: %s on %s", p, s.ServerName)
+				}
+			}()
+			osTypeChan <- detectOS(s)
 		}(s)
 	}
 
-	timeout := time.After(60 * time.Second)
+	var oses []osTypeInterface
+	timeout := time.After(30 * time.Second)
 	for i := 0; i < len(config.Conf.Servers); i++ {
 		select {
 		case res := <-osTypeChan:
-			Log.Infof("(%d/%d) Successfully detected. %s: %s",
-				i+1, len(config.Conf.Servers),
-				res.getServerInfo().ServerName,
-				res.getDistributionInfo())
-			osi = append(osi, res)
+			oses = append(oses, res)
+			if 0 < len(res.getErrs()) {
+				Log.Errorf("(%d/%d) Failed: %s, err: %s",
+					i+1, len(config.Conf.Servers),
+					res.getServerInfo().ServerName,
+					res.getErrs())
+			} else {
+				Log.Infof("(%d/%d) Detected: %s: %s",
+					i+1, len(config.Conf.Servers),
+					res.getServerInfo().ServerName,
+					res.getDistro())
+			}
 		case <-timeout:
-			Log.Error("Timeout occured while detecting")
-			err = fmt.Errorf("Timeout")
+			msg := "Timed out while detecting servers"
+			Log.Error(msg)
 			for servername := range config.Conf.Servers {
 				found := false
-				for _, o := range osi {
+				for _, o := range oses {
 					if servername == o.getServerInfo().ServerName {
 						found = true
 						break
 					}
 				}
 				if !found {
-					Log.Errorf("Failed to detect. servername: %s", servername)
+					Log.Errorf("(%d/%d) Timed out: %s",
+						i+1, len(config.Conf.Servers),
+						servername)
+					i++
 				}
 			}
-			return
+		}
+	}
+
+	for _, o := range oses {
+		if len(o.getErrs()) == 0 {
+			sshAbleOses = append(sshAbleOses, o)
 		}
 	}
 	return
 }
 
+func detectContainerOSes() (actives []osTypeInterface) {
+	Log.Info("Detecting OS of containers... ")
+	osTypesChan := make(chan []osTypeInterface, len(servers))
+	defer close(osTypesChan)
+	for _, s := range servers {
+		go func(s osTypeInterface) {
+			defer func() {
+				if p := recover(); p != nil {
+					Log.Debugf("Panic: %s on %s",
+						p, s.getServerInfo().GetServerName())
+				}
+			}()
+			osTypesChan <- detectContainerOSesOnServer(s)
+		}(s)
+	}
+
+	var oses []osTypeInterface
+	timeout := time.After(30 * time.Second)
+	for i := 0; i < len(servers); i++ {
+		select {
+		case res := <-osTypesChan:
+			for _, osi := range res {
+				sinfo := osi.getServerInfo()
+				if 0 < len(osi.getErrs()) {
+					Log.Errorf("Failed: %s err: %s", sinfo.ServerName, osi.getErrs())
+					continue
+				}
+				oses = append(oses, res...)
+				Log.Infof("Detected: %s@%s: %s",
+					sinfo.Container.Name, sinfo.ServerName, osi.getDistro())
+			}
+		case <-timeout:
+			msg := "Timed out while detecting containers"
+			Log.Error(msg)
+			for servername := range config.Conf.Servers {
+				found := false
+				for _, o := range oses {
+					if servername == o.getServerInfo().ServerName {
+						found = true
+						break
+					}
+				}
+				if !found {
+					Log.Errorf("Timed out: %s", servername)
+
+				}
+			}
+		}
+	}
+	for _, o := range oses {
+		if len(o.getErrs()) == 0 {
+			actives = append(actives, o)
+		}
+	}
+	return
+}
+
+func detectContainerOSesOnServer(containerHost osTypeInterface) (oses []osTypeInterface) {
+	containerHostInfo := containerHost.getServerInfo()
+	if len(containerHostInfo.Containers) == 0 {
+		return
+	}
+
+	running, err := containerHost.runningContainers()
+	if err != nil {
+		containerHost.setErrs([]error{fmt.Errorf(
+			"Failed to get running containers on %s. err: %s",
+			containerHost.getServerInfo().ServerName, err)})
+		return append(oses, containerHost)
+	}
+
+	if containerHostInfo.Containers[0] == "${running}" {
+		for _, containerInfo := range running {
+			copied := containerHostInfo
+			copied.SetContainer(config.Container{
+				ContainerID: containerInfo.ContainerID,
+				Name:        containerInfo.Name,
+			})
+			os := detectOS(copied)
+			oses = append(oses, os)
+		}
+		return oses
+	}
+
+	exitedContainers, err := containerHost.exitedContainers()
+	if err != nil {
+		containerHost.setErrs([]error{fmt.Errorf(
+			"Failed to get exited containers on %s. err: %s",
+			containerHost.getServerInfo().ServerName, err)})
+		return append(oses, containerHost)
+	}
+
+	var exited, unknown []string
+	for _, container := range containerHostInfo.Containers {
+		found := false
+		for _, c := range running {
+			if c.ContainerID == container || c.Name == container {
+				copied := containerHostInfo
+				copied.SetContainer(c)
+				os := detectOS(copied)
+				oses = append(oses, os)
+				found = true
+				break
+			}
+		}
+
+		if !found {
+			foundInExitedContainers := false
+			for _, c := range exitedContainers {
+				if c.ContainerID == container || c.Name == container {
+					exited = append(exited, container)
+					foundInExitedContainers = true
+					break
+				}
+			}
+			if !foundInExitedContainers {
+				unknown = append(unknown, container)
+			}
+		}
+	}
+	if 0 < len(exited) || 0 < len(unknown) {
+		containerHost.setErrs([]error{fmt.Errorf(
+			"Some containers on %s are exited or unknown. exited: %s, unknown: %s",
+			containerHost.getServerInfo().ServerName, exited, unknown)})
+		return append(oses, containerHost)
+	}
+	return oses
+}
+
+// CheckIfSudoNoPasswd checks whether vuls can sudo with nopassword via SSH
+func CheckIfSudoNoPasswd(localLogger *logrus.Entry) error {
+	timeoutSec := 15
+	errs := parallelSSHExec(func(o osTypeInterface) error {
+		return o.checkIfSudoNoPasswd()
+	}, timeoutSec)
+
+	if 0 < len(errs) {
+		return fmt.Errorf(fmt.Sprintf("%s", errs))
+	}
+	return nil
+}
+
+// DetectPlatforms detects the platform of each servers.
+func DetectPlatforms(localLogger *logrus.Entry) {
+	errs := detectPlatforms()
+	if 0 < len(errs) {
+		// Only logging
+		Log.Warnf("Failed to detect platforms. err: %v", errs)
+	}
+	for i, s := range servers {
+		if s.getServerInfo().IsContainer() {
+			Log.Infof("(%d/%d) %s on %s is running on %s",
+				i+1, len(servers),
+				s.getServerInfo().Container.Name,
+				s.getServerInfo().ServerName,
+				s.getPlatform().Name,
+			)
+
+		} else {
+			Log.Infof("(%d/%d) %s is running on %s",
+				i+1, len(servers),
+				s.getServerInfo().ServerName,
+				s.getPlatform().Name,
+			)
+		}
+	}
+	return
+}
+
+func detectPlatforms() []error {
+	timeoutSec := 1 * 60
+	return parallelSSHExec(func(o osTypeInterface) error {
+		return o.detectPlatform()
+	}, timeoutSec)
+}
+
 // Prepare installs requred packages to scan vulnerabilities.
 func Prepare() []error {
-	return parallelSSHExec(func(o osTypeInterface) error {
+	errs := parallelSSHExec(func(o osTypeInterface) error {
+		if err := o.checkDependencies(); err != nil {
+			return err
+		}
+		return nil
+	})
+	if len(errs) != 0 {
+		return errs
+	}
+
+	var targets []osTypeInterface
+	for _, s := range servers {
+		deps := s.getLackDependencies()
+		if len(deps) != 0 {
+			targets = append(targets, s)
+		}
+	}
+	if len(targets) == 0 {
+		Log.Info("No need to install dependencies")
+		return nil
+	}
+
+	Log.Info("Below servers are needed to install dependencies")
+	for _, s := range targets {
+		for _, d := range s.getLackDependencies() {
+			Log.Infof("  - %s on %s", d, s.getServerInfo().GetServerName())
+		}
+	}
+	Log.Info("Is this ok to install dependencies on the servers? [y/N]")
+
+	reader := bufio.NewReader(os.Stdin)
+	for {
+		text, err := reader.ReadString('\n')
+		if err != nil {
+			return []error{err}
+		}
+		switch strings.TrimSpace(text) {
+		case "", "N", "n":
+			return nil
+		case "y", "Y":
+			goto yes
+		default:
+			Log.Info("Please enter y or N")
+		}
+	}
+
+yes:
+	servers = targets
+	errs = parallelSSHExec(func(o osTypeInterface) error {
 		if err := o.install(); err != nil {
 			return err
 		}
 		return nil
 	})
+	if len(errs) != 0 {
+		return errs
+	}
+	Log.Info("All dependencies were installed correctly")
+	return nil
 }
 
 // Scan scan
@@ -174,6 +504,16 @@ func Scan() []error {
 		return errs
 	}
 
+	if err := setupCangelogCache(); err != nil {
+		return []error{err}
+	}
+
+	defer func() {
+		if cache.DB != nil {
+			cache.DB.Close()
+		}
+	}()
+
 	Log.Info("Scanning vulnerable OS packages...")
 	if errs := scanPackages(); errs != nil {
 		return errs
@@ -186,6 +526,23 @@ func Scan() []error {
 	return nil
 }
 
+func setupCangelogCache() error {
+	needToSetupCache := false
+	for _, s := range servers {
+		switch s.getDistro().Family {
+		case "ubuntu", "debian":
+			needToSetupCache = true
+			break
+		}
+	}
+	if needToSetupCache {
+		if err := cache.SetupBolt(config.Conf.CacheDBPath, Log); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
 func checkRequiredPackagesInstalled() []error {
 	timeoutSec := 30 * 60
 	return parallelSSHExec(func(o osTypeInterface) error {
@@ -194,7 +551,7 @@ func checkRequiredPackagesInstalled() []error {
 }
 
 func scanPackages() []error {
-	timeoutSec := 30 * 60
+	timeoutSec := 120 * 60
 	return parallelSSHExec(func(o osTypeInterface) error {
 		return o.scanPackages()
 	}, timeoutSec)

scan/sshutil.go

@@ -25,7 +25,10 @@ import (
 	"io/ioutil"
 	"net"
 	"os"
+	"os/exec"
+	"runtime"
 	"strings"
+	"syscall"
 	"time"
 
 	"golang.org/x/crypto/ssh"
@@ -34,18 +37,30 @@ import (
 	"github.com/Sirupsen/logrus"
 	"github.com/cenkalti/backoff"
 	conf "github.com/future-architect/vuls/config"
-	"github.com/k0kubun/pp"
+	"github.com/future-architect/vuls/util"
 )
 
 type sshResult struct {
+	Servername string
 	Host       string
 	Port       string
+	Cmd        string
 	Stdout     string
 	Stderr     string
 	ExitStatus int
+	Error      error
+}
+
+func (s sshResult) String() string {
+	return fmt.Sprintf(
+		"SSHResult: servername: %s, cmd: %s, exitstatus: %d, stdout: %s, stderr: %s, err: %s",
+		s.Servername, s.Cmd, s.ExitStatus, s.Stdout, s.Stderr, s.Error)
 }
 
 func (s sshResult) isSuccess(expectedStatusCodes ...int) bool {
+	if s.Error != nil {
+		return false
+	}
 	if len(expectedStatusCodes) == 0 {
 		return s.ExitStatus == 0
 	}
@@ -64,10 +79,19 @@ const sudo = true
 const noSudo = false
 
 func parallelSSHExec(fn func(osTypeInterface) error, timeoutSec ...int) (errs []error) {
+	resChan := make(chan string, len(servers))
 	errChan := make(chan error, len(servers))
 	defer close(errChan)
+	defer close(resChan)
+
 	for _, s := range servers {
 		go func(s osTypeInterface) {
+			defer func() {
+				if p := recover(); p != nil {
+					logrus.Debugf("Panic: %s on %s",
+						p, s.getServerInfo().GetServerName())
+				}
+			}()
 			if err := fn(s); err != nil {
 				errChan <- fmt.Errorf("%s@%s:%s: %s",
 					s.getServerInfo().User,
@@ -76,7 +100,7 @@ func parallelSSHExec(fn func(osTypeInterface) error, timeoutSec ...int) (errs []
 					err,
 				)
 			} else {
-				errChan <- nil
+				resChan <- s.getServerInfo().GetServerName()
 			}
 		}(s)
 	}
@@ -88,67 +112,78 @@ func parallelSSHExec(fn func(osTypeInterface) error, timeoutSec ...int) (errs []
 		timeout = timeoutSec[0]
 	}
 
+	var snames []string
+	isTimedout := false
 	for i := 0; i < len(servers); i++ {
 		select {
+		case s := <-resChan:
+			snames = append(snames, s)
 		case err := <-errChan:
-			if err != nil {
-				errs = append(errs, err)
-			} else {
-				logrus.Debug("Parallel SSH Success")
-			}
+			errs = append(errs, err)
 		case <-time.After(time.Duration(timeout) * time.Second):
-			logrus.Errorf("Parallel SSH Timeout")
-			errs = append(errs, fmt.Errorf("Timed out"))
+			isTimedout = true
 		}
 	}
+
+	// collect timed out servernames
+	var timedoutSnames []string
+	if isTimedout {
+		for _, s := range servers {
+			name := s.getServerInfo().GetServerName()
+			found := false
+			for _, t := range snames {
+				if name == t {
+					found = true
+					break
+				}
+			}
+			if !found {
+				timedoutSnames = append(timedoutSnames, name)
+			}
+		}
+	}
+	if isTimedout {
+		errs = append(errs, fmt.Errorf(
+			"Timed out: %s", timedoutSnames))
+	}
 	return
 }
 
 func sshExec(c conf.ServerInfo, cmd string, sudo bool, log ...*logrus.Entry) (result sshResult) {
-	// Setup Logger
-	var logger *logrus.Entry
-	if len(log) == 0 {
-		level := logrus.InfoLevel
-		if conf.Conf.Debug == true {
-			level = logrus.DebugLevel
-		}
-		l := &logrus.Logger{
-			Out:       os.Stderr,
-			Formatter: new(logrus.TextFormatter),
-			Hooks:     make(logrus.LevelHooks),
-			Level:     level,
-		}
-		logger = logrus.NewEntry(l)
+	if isSSHExecNative() {
+		result = sshExecNative(c, cmd, sudo)
 	} else {
-		logger = log[0]
+		result = sshExecExternal(c, cmd, sudo)
 	}
 
-	var err error
-	if sudo && c.User != "root" {
-		switch {
-		case c.SudoOpt.ExecBySudo:
-			cmd = fmt.Sprintf("echo %s | sudo -S %s", c.Password, cmd)
-		case c.SudoOpt.ExecBySudoSh:
-			cmd = fmt.Sprintf("echo %s | sudo sh -c '%s'", c.Password, cmd)
-		default:
-			logger.Panicf("sudoOpt is invalid. SudoOpt: %v", c.SudoOpt)
-		}
-	}
-	// set pipefail option.
-	// http://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
-	cmd = fmt.Sprintf("set -o pipefail; %s", cmd)
-	logger.Debugf("Command: %s",
-		strings.Replace(maskPassword(cmd, c.Password), "\n", "", -1))
+	logger := getSSHLogger(log...)
+	logger.Debug(result)
+	return
+}
+
+func isSSHExecNative() bool {
+	return runtime.GOOS == "windows" || !conf.Conf.SSHExternal
+}
+
+func sshExecNative(c conf.ServerInfo, cmd string, sudo bool) (result sshResult) {
+	result.Servername = c.ServerName
+	result.Host = c.Host
+	result.Port = c.Port
 
 	var client *ssh.Client
-	client, err = sshConnect(c)
+	var err error
+	if client, err = sshConnect(c); err != nil {
+		result.Error = err
+		result.ExitStatus = 999
+		return
+	}
 	defer client.Close()
 
 	var session *ssh.Session
 	if session, err = client.NewSession(); err != nil {
-		logger.Errorf("Failed to new session. err: %s, c: %s",
-			err,
-			pp.Sprintf("%v", c))
+		result.Error = fmt.Errorf(
+			"Failed to create a new session. servername: %s, err: %s",
+			c.ServerName, err)
 		result.ExitStatus = 999
 		return
 	}
@@ -160,11 +195,10 @@ func sshExec(c conf.ServerInfo, cmd string, sudo bool, log ...*logrus.Entry) (re
 		ssh.TTY_OP_ISPEED: 14400, // input speed = 14.4kbaud
 		ssh.TTY_OP_OSPEED: 14400, // output speed = 14.4kbaud
 	}
-	if err = session.RequestPty("xterm", 400, 120, modes); err != nil {
-		logger.Errorf("Failed to request for pseudo terminal. err: %s, c: %s",
-			err,
-			pp.Sprintf("%v", c))
-
+	if err = session.RequestPty("xterm", 400, 256, modes); err != nil {
+		result.Error = fmt.Errorf(
+			"Failed to request for pseudo terminal. servername: %s, err: %s",
+			c.ServerName, err)
 		result.ExitStatus = 999
 		return
 	}
@@ -173,6 +207,7 @@ func sshExec(c conf.ServerInfo, cmd string, sudo bool, log ...*logrus.Entry) (re
 	session.Stdout = &stdoutBuf
 	session.Stderr = &stderrBuf
 
+	cmd = decolateCmd(c, cmd, sudo)
 	if err := session.Run(cmd); err != nil {
 		if exitErr, ok := err.(*ssh.ExitError); ok {
 			result.ExitStatus = exitErr.ExitStatus()
@@ -185,18 +220,106 @@ func sshExec(c conf.ServerInfo, cmd string, sudo bool, log ...*logrus.Entry) (re
 
 	result.Stdout = stdoutBuf.String()
 	result.Stderr = stderrBuf.String()
-	result.Host = c.Host
-	result.Port = c.Port
-
-	logger.Debugf(
-		"SSH executed. cmd: %s, status: %#v\nstdout: \n%s\nstderr: \n%s",
-		maskPassword(cmd, c.Password), err, result.Stdout, result.Stderr)
-
+	result.Cmd = strings.Replace(cmd, "\n", "", -1)
 	return
 }
 
+func sshExecExternal(c conf.ServerInfo, cmd string, sudo bool) (result sshResult) {
+	sshBinaryPath, err := exec.LookPath("ssh")
+	if err != nil {
+		return sshExecNative(c, cmd, sudo)
+	}
+
+	defaultSSHArgs := []string{
+		"-t",
+		"-o", "StrictHostKeyChecking=no",
+		"-o", "UserKnownHostsFile=/dev/null",
+		"-o", "LogLevel=quiet",
+		"-o", "ConnectionAttempts=3",
+		"-o", "ConnectTimeout=10",
+		"-o", "ControlMaster=no",
+		"-o", "ControlPath=none",
+
+		// TODO ssh session multiplexing
+		//  "-o", "ControlMaster=auto",
+		//  "-o", `ControlPath=~/.ssh/controlmaster-%r-%h.%p`,
+		//  "-o", "Controlpersist=30m",
+	}
+	args := append(defaultSSHArgs, fmt.Sprintf("%s@%s", c.User, c.Host))
+	args = append(args, "-p", c.Port)
+
+	//  if conf.Conf.Debug {
+	//      args = append(args, "-v")
+	//  }
+
+	if 0 < len(c.KeyPath) {
+		args = append(args, "-i", c.KeyPath)
+		args = append(args, "-o", "PasswordAuthentication=no")
+	}
+
+	cmd = decolateCmd(c, cmd, sudo)
+	//  cmd = fmt.Sprintf("stty cols 256; set -o pipefail; %s", cmd)
+
+	args = append(args, cmd)
+	execCmd := exec.Command(sshBinaryPath, args...)
+
+	var stdoutBuf, stderrBuf bytes.Buffer
+	execCmd.Stdout = &stdoutBuf
+	execCmd.Stderr = &stderrBuf
+	if err := execCmd.Run(); err != nil {
+		if e, ok := err.(*exec.ExitError); ok {
+			if s, ok := e.Sys().(syscall.WaitStatus); ok {
+				result.ExitStatus = s.ExitStatus()
+			} else {
+				result.ExitStatus = 998
+			}
+		} else {
+			result.ExitStatus = 999
+		}
+	} else {
+		result.ExitStatus = 0
+	}
+
+	result.Stdout = stdoutBuf.String()
+	result.Stderr = stderrBuf.String()
+	result.Servername = c.ServerName
+	result.Host = c.Host
+	result.Port = c.Port
+	result.Cmd = fmt.Sprintf("%s %s", sshBinaryPath, strings.Join(args, " "))
+	return
+}
+
+func getSSHLogger(log ...*logrus.Entry) *logrus.Entry {
+	if len(log) == 0 {
+		return util.NewCustomLogger(conf.ServerInfo{})
+	}
+	return log[0]
+}
+
+func decolateCmd(c conf.ServerInfo, cmd string, sudo bool) string {
+	if sudo && c.User != "root" && !c.IsContainer() {
+		cmd = fmt.Sprintf("sudo -S %s", cmd)
+		cmd = strings.Replace(cmd, "|", "| sudo ", -1)
+	}
+
+	if c.Distro.Family != "FreeBSD" {
+		// set pipefail option. Bash only
+		// http://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another
+		cmd = fmt.Sprintf("set -o pipefail; %s", cmd)
+	}
+
+	if c.IsContainer() {
+		switch c.Container.Type {
+		case "", "docker":
+			cmd = fmt.Sprintf(`docker exec %s /bin/bash -c "%s"`, c.Container.ContainerID, cmd)
+		}
+	}
+	//  cmd = fmt.Sprintf("set -x; %s", cmd)
+	return cmd
+}
+
 func getAgentAuth() (auth ssh.AuthMethod, ok bool) {
-	if sock := os.Getenv("SSH_AUTH_SOCK"); len(sock) > 0 {
+	if sock := os.Getenv("SSH_AUTH_SOCK"); 0 < len(sock) {
 		if agconn, err := net.Dial("unix", sock); err == nil {
 			ag := agent.NewClient(agconn)
 			auth = ssh.PublicKeysCallback(ag.Signers)
@@ -219,19 +342,13 @@ func tryAgentConnect(c conf.ServerInfo) *ssh.Client {
 }
 
 func sshConnect(c conf.ServerInfo) (client *ssh.Client, err error) {
-
 	if client = tryAgentConnect(c); client != nil {
 		return client, nil
 	}
 
 	var auths = []ssh.AuthMethod{}
 	if auths, err = addKeyAuth(auths, c.KeyPath, c.KeyPassword); err != nil {
-		logrus.Fatalf("Failed to add keyAuth. %s@%s:%s err: %s",
-			c.User, c.Host, c.Port, err)
-	}
-
-	if c.Password != "" {
-		auths = append(auths, ssh.Password(c.Password))
+		return nil, err
 	}
 
 	// http://blog.ralch.com/tutorial/golang-ssh-connection/
@@ -241,8 +358,9 @@ func sshConnect(c conf.ServerInfo) (client *ssh.Client, err error) {
 	}
 
 	notifyFunc := func(e error, t time.Duration) {
-		logrus.Warnf("Failed to ssh %s@%s:%s err: %s, Retrying in %s...",
-			c.User, c.Host, c.Port, e, t)
+		logger := getSSHLogger()
+		logger.Debugf("Failed to Dial to %s, err: %s, Retrying in %s...",
+			c.ServerName, e, t)
 	}
 	err = backoff.RetryNotify(func() error {
 		if client, err = ssh.Dial("tcp", c.Host+":"+c.Port, config); err != nil {
@@ -306,11 +424,6 @@ func parsePemBlock(block *pem.Block) (interface{}, error) {
 	case "DSA PRIVATE KEY":
 		return ssh.ParseDSAPrivateKey(block.Bytes)
 	default:
-		return nil, fmt.Errorf("rtop: unsupported key type %q", block.Type)
+		return nil, fmt.Errorf("Unsupported key type %q", block.Type)
 	}
 }
-
-// ref golang.org/x/crypto/ssh/keys.go#ParseRawPrivateKey.
-func maskPassword(cmd, sudoPass string) string {
-	return strings.Replace(cmd, fmt.Sprintf("echo %s", sudoPass), "echo *****", -1)
-}

setup/docker/README.md

@@ -0,0 +1,183 @@
+# Vuls Docker components
+
+This is the Git repo of the official Docker image for vuls.
+
+# Supported tags and respective `Dockerfile` links
+
+- go-cve-dictionary
+  - [`latest` (*go-cve-dictionary:latest Dockerfile*)]()
+- vuls
+  - [`latest` (*vuls:latest Dockerfile*)]()
+- vulsrepo
+  - [`latest` (*vulsrepo:latest Dockerfile*)]()
+
+This image version is same as the github repository version.
+
+# Caution
+
+This image is built per commit.
+If you want to use the latest docker image, you should remove the existing image, and pull it once again.
+
+1. Confirm your vuls version
+
+- go-cve-dictionary
+
+```console
+$ docker run  --rm  vuls/go-cve-dictionary -v
+
+go-cve-dictionary v0.0.xxx xxxx
+```
+
+- vuls
+
+```console
+$ docker run  --rm  vuls/vuls -v
+
+vuls v0.0.xxx xxxx
+```
+
+2. Remove your old docker images
+
+- go-cve-dictionary
+
+```
+$ docker rmi vuls/go-cve-dictionary
+```
+
+```
+$ docker rmi vuls/vuls
+```
+
+- vuls
+
+```
+$ docker rmi vuls/vuls
+```
+
+3. Pull new vuls docker images
+
+- go-cve-dictionary
+
+```
+$ docker pull vuls/go-cve-dictionary
+```
+
+- vuls
+
+```
+$ docker pull vuls/vuls
+```
+
+4. Confirm your vuls version
+
+```console
+$ docker run  --rm  vuls/go-cve-dictionary -v
+
+go-cve-dictionary v0.1.xxx xxxx
+```
+
+- vuls
+
+```console
+$ docker run  --rm  vuls/vuls -v
+
+vuls v0.1.xxx xxxx
+```
+
+
+# How to use this image
+
+1. fetch nvd (vuls/go-cve-dictionary)
+1. configuration (vuls/vuls)
+1. prepare (vuls/vuls)
+1. scan (vuls/vuls)
+1. vulsrepo (vuls/vulsrepo)
+
+## Step1. Fetch NVD
+
+```console
+$ for i in {2002..2016}; do \
+    docker run --rm -it \
+    -v $PWD:/vuls \
+    -v $PWD/go-cve-dictionary-log:/var/log/vuls \
+    vuls/go-cve-dictionary fetchnvd -years $i; \
+  done
+```
+
+## Step2. Configuration
+
+Create config.toml referring to [this](https://github.com/future-architect/vuls#configuration).
+
+```toml
+[servers]
+
+[servers.amazon]
+host         = "54.249.93.16"
+port        = "22"
+user        = "vuls-user"
+keyPath     = "/root/.ssh/id_rsa" # path to ssh private key in docker
+```
+　
+
+```console
+$ docker run --rm \
+    -v ~/.ssh:/root/.ssh:ro \
+    -v $PWD:/vuls \
+    -v $PWD/vuls-log:/var/log/vuls \
+    vuls/vuls configtest \
+    -config=./config.toml # path to config.toml in docker
+```
+
+## Step3. Prepare
+
+```console
+$ docker run --rm \
+    -v ~/.ssh:/root/.ssh:ro \
+    -v $PWD:/vuls \
+    -v $PWD/vuls-log:/var/log/vuls \
+    vuls/vuls prepare \
+    -config=./config.toml # path to config.toml in docker
+```
+
+## Step4. Scan
+
+```console
+$ docker run --rm -it \
+    -v ~/.ssh:/root/.ssh:ro \
+    -v $PWD:/vuls \
+    -v $PWD/vuls-log:/var/log/vuls \
+    -v /etc/localtime:/etc/localtime:ro \
+    -e "TZ=Asia/Tokyo" \
+    vuls/vuls scan \
+    -cve-dictionary-dbpath=/vuls/cve.sqlite3 \
+    -report-json \
+    -config=./config.toml # path to config.toml in docker
+```
+
+## Step5. vulsrepo
+
+```console
+$docker run -dt \
+    -v $PWD:/vuls \
+    -p 80:80 \
+    vuls/vulsrepo
+```
+
+# User Feedback
+
+## Documentation
+
+Documentation for this image is stored in the [`docker/` directory]() of the [`future-architect/vuls` GitHub repo](https://github.com/future-architect/vuls). 
+
+## Issues
+
+If you have any problems with or questions about this image, please contact us through a [GitHub issue](https://github.com/future-architect/vuls/issues). 
+
+## Contributing
+
+1. fork a repository: github.com/future-architect/vuls to github.com/you/repo
+1. get original code: go get github.com/future-architect/vuls
+1. work on original code
+1. add remote to your repo: git remote add myfork https://github.com/you/repo.git
+1. push your changes: git push myfork
+1. create a new Pull Request

setup/docker/go-cve-dictionary/latest/Dockerfile

@@ -0,0 +1,19 @@
+FROM golang:latest
+
+MAINTAINER hikachan sadayuki-matsuno
+
+ENV REPOSITORY github.com/kotakanbe/go-cve-dictionary
+ENV LOGDIR /var/log/vuls
+ENV WORKDIR /vuls
+# go-cve-dictionary install
+RUN git clone https://$REPOSITORY.git $GOPATH/src/$REPOSITORY \
+    && cd $GOPATH/src/$REPOSITORY \
+    && make install \
+    && mkdir -p $LOGDIR
+
+VOLUME [$WORKDIR, $LOGDIR]
+WORKDIR $WORKDIR
+ENV PWD $WORKDIR
+
+ENTRYPOINT ["go-cve-dictionary"]
+CMD ["--help"]

setup/docker/go-cve-dictionary/latest/README.md

@@ -0,0 +1,89 @@
+# go-cve-dictionary-Docker
+
+This is the Git repo of the official Docker image for go-cve-dictionary.
+See the [Hub page](https://hub.docker.com/r/vuls/go-cve-dictionary/) for the full readme on how to use the Docker image and for information regarding contributing and issues.
+
+# Supported tags and respective `Dockerfile` links
+
+- [`latest` (*go-cve-dictionary:latest Dockerfile*)](https://github.com/future-architect/vuls/blob/master/setup/docker/go-cve-dictionary/latest/Dockerfile)
+
+# Caution
+
+This image is built per commit.
+If you want to use the latest docker image, you should remove the existing image, and pull it once again.
+
+- Remove old docker image
+
+```
+$ docker rmi vuls/go-cve-dictionary
+```
+
+- Pull new docker image
+
+```
+$ docker pull vuls/go-cve-dictionary
+```
+
+# What is go-cve-dictionary?
+
+This is tool to build a local copy of the NVD (National Vulnerabilities Database) [1] and the Japanese JVN [2], which contain security vulnerabilities according to their CVE identifiers [3] including exhaustive information and a risk score. The local copy is generated in sqlite format, and the tool has a server mode for easy querying.
+
+[1] https://en.wikipedia.org/wiki/National_Vulnerability_Database  
+[2] https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures  
+[3] http://jvndb.jvn.jp/apis/termsofuse.html  
+
+# How to use this image
+
+## check vuls version
+
+```
+$ docker run --rm vuls/go-cve-dictionary -v
+```
+
+## fetchnvd
+
+```console
+$ for i in {2002..2016}; do \
+    docker run --rm -it \
+    -v $PWD:/vuls \
+    -v $PWD/go-cve-dictionary-log:/var/log/vuls \
+    vuls/go-cve-dictionary fetchnvd -years $i; \
+  done
+```
+
+## server
+
+```console
+$ docker run -dt \
+    --name go-cve-dictionary \
+    -v $PWD:/vuls \
+    -v $PWD/go-cve-dictionary-log:/var/log/vuls \
+    --expose 1323 \
+    -p 1323:1323 \
+    vuls/go-cve-dictionary server --bind=0.0.0.0
+```
+
+Prease refer to [this](https://hub.docker.com/r/vuls/go-cve-dictionary).
+
+## vuls
+
+Please refer to [this](https://hub.docker.com/r/vuls/vuls/).
+
+# User Feedback
+
+## Documentation
+
+Documentation for this image is stored in the [`docker/` directory](https://github.com/future-architect/vuls/tree/master/setup/docker) of the [`future-architect/vuls` GitHub repo](https://github.com/future-architect/vuls). 
+
+## Issues
+
+If you have any problems with or questions about this image, please contact us through a [GitHub issue](https://github.com/future-architect/vuls/issues). 
+
+## Contributing
+
+1. fork a repository: github.com/future-architect/vuls to github.com/you/repo
+1. get original code: go get github.com/future-architect/vuls
+1. work on original code
+1. add remote to your repo: git remote add myfork https://github.com/you/repo.git
+1. push your changes: git push myfork
+1. create a new Pull Request

setup/docker/vuls/latest/Dockerfile

@@ -0,0 +1,19 @@
+FROM golang:latest
+
+MAINTAINER hikachan sadayuki-matsuno
+
+ENV REPOSITORY github.com/future-architect/vuls
+ENV LOGDIR /var/log/vuls
+ENV WORKDIR /vuls
+# go-cve-dictionary install
+RUN git clone https://$REPOSITORY.git $GOPATH/src/$REPOSITORY \
+    && cd $GOPATH/src/$REPOSITORY \
+    && make install \
+    && mkdir -p $LOGDIR
+
+VOLUME [$WORKDIR, $LOGDIR]
+WORKDIR $WORKDIR
+ENV PWD $WORKDIR
+
+ENTRYPOINT ["vuls"]
+CMD ["--help"]

setup/docker/vuls/latest/README.md

@@ -0,0 +1,121 @@
+# Vuls-Docker
+
+This is the Git repo of the official Docker image for vuls.
+See the [Hub page](https://hub.docker.com/r/vuls/vuls/) for the full readme on how to use the Docker image and for information regarding contributing and issues.
+
+# Supported tags and respective `Dockerfile` links
+
+- [`latest` (*vuls:latest Dockerfile*)](https://github.com/future-architect/vuls/blob/master/setup/docker/vuls/latest/Dockerfile)
+
+# Caution
+
+This image is built per commit.
+If you want to use the latest docker image, you should remove the existing image, and pull it once again.
+
+- Remove old docker image
+
+```
+$ docker rmi vuls/vuls
+```
+
+- Pull new docker image
+
+```
+$ docker pull vuls/vuls
+```
+
+# What is Vuls?
+
+Vuls is the Vulnerability scanner for Linux/FreeBSD, agentless, written in golang.
+Please see the [Documentation](https://github.com/future-architect/vuls)
+
+![logo](https://github.com/future-architect/vuls/blob/master/img/vuls_logo.png?raw=true)
+
+# How to use this image
+
+## check vuls version
+
+```
+$ docker run  --rm  vuls/vuls -v
+```
+
+## configtest
+
+Create config.toml referring to [this](https://github.com/future-architect/vuls#configuration).
+
+```toml
+[servers]
+
+[servers.amazon]
+host         = "54.249.93.16"
+port        = "22"
+user        = "vuls-user"
+keyPath     = "/root/.ssh/id_rsa"  # path to ssh private key in docker
+```
+　
+
+```console
+$ docker run --rm \
+    -v ~/.ssh:/root/.ssh:ro \
+    -v $PWD:/vuls \
+    -v $PWD/vuls-log:/var/log/vuls \
+    vuls/vuls configtest
+```
+
+
+## prepare
+
+```console
+$ docker run --rm \
+    -v ~/.ssh:/root/.ssh:ro \
+    -v $PWD:/vuls \
+    -v $PWD/vuls-log:/var/log/vuls \
+    vuls/vuls prepare \
+    -config=./config.toml # path to config.toml in docker
+```
+
+## scan
+
+```console
+$ docker run --rm -it \
+    -v ~/.ssh:/root/.ssh:ro \
+    -v $PWD:/vuls \
+    -v $PWD/vuls-log:/var/log/vuls \
+    -v /etc/localtime:/etc/localtime:ro \
+    vuls/vuls scan \
+    -cve-dictionary-dbpath=/vuls/cve.sqlite3 \
+    -config=./config.toml \ # path to config.toml in docker
+    -report-json 
+```
+
+## tui
+
+```console
+$ docker run --rm -it \
+    -v $PWD:/vuls \
+    -v $PWD/vuls-log:/var/log/vuls \
+    vuls/vuls tui 
+```
+
+## vulsrepo
+
+Prease refer to [this](https://hub.docker.com/r/vuls/vulsrepo/).
+
+# User Feedback
+
+## Documentation
+
+Documentation for this image is stored in the [`docker/` directory](https://github.com/future-architect/vuls/tree/master/setup/docker) of the [`future-architect/vuls` GitHub repo](https://github.com/future-architect/vuls). 
+
+## Issues
+
+If you have any problems with or questions about this image, please contact us through a [GitHub issue](https://github.com/future-architect/vuls/issues). 
+
+## Contributing
+
+1. fork a repository: github.com/future-architect/vuls to github.com/you/repo
+1. get original code: go get github.com/future-architect/vuls
+1. work on original code
+1. add remote to your repo: git remote add myfork https://github.com/you/repo.git
+1. push your changes: git push myfork
+1. create a new Pull Request

setup/docker/vulsrepo/latest/Dockerfile

@@ -0,0 +1,31 @@
+FROM httpd:2.4
+
+MAINTAINER hikachan sadayuki-matsuno
+# install packages
+RUN apt-get update \
+        && apt-get install -y --no-install-recommends \
+      		ca-certificates \
+		      vim \
+          git \
+          libcgi-pm-perl \
+          libjson-perl \
+        && rm -r /var/lib/apt/lists/*
+
+# env
+ENV HTTPD_PREFIX /usr/local/apache2
+
+VOLUME /vuls
+
+WORKDIR ${HTTPD_PREFIX}/htdocs
+RUN git clone https://github.com/usiusi360/vulsrepo.git \
+     && echo "LoadModule cgid_module modules/mod_cgid.so" >> $HTTPD_PREFIX/conf/httpd.conf \
+     && echo "<Directory \"$HTTPD_PREFIX/htdocs/vulsrepo/dist/cgi\">" >> $HTTPD_PREFIX/conf/httpd.conf \
+     && echo "  Options +ExecCGI +FollowSymLinks" >> $HTTPD_PREFIX/conf/httpd.conf \
+     && echo "  AddHandler cgi-script cgi" >> $HTTPD_PREFIX/conf/httpd.conf \
+     && echo "</Directory>" >> $HTTPD_PREFIX/conf/httpd.conf \
+     && sed -i -e 's/User daemon/#User/g' $HTTPD_PREFIX/conf/httpd.conf \
+     && sed -i -e 's/Group daemon/#Group/g' $HTTPD_PREFIX/conf/httpd.conf \
+     && ln -snf /vuls/results /usr/local/apache2/htdocs/vulsrepo/results
+
+EXPOSE 80
+CMD ["httpd-foreground"]

setup/docker/vulsrepo/latest/README.md

@@ -0,0 +1,47 @@
+# VulsRepo-Docker
+
+This is the Git repo of the official Docker image for vulsrepo.
+See the [Hub page](https://hub.docker.com/r/vuls/vulsrepo/) for the full readme on how to use the Docker image and for information regarding contributing and issues.
+
+# Supported tags and respective `Dockerfile` links
+
+- [`latest` (*vulsrepo:latest Dockerfile*)](https://github.com/future-architect/vuls/blob/master/setup/docker/vulsrepo/latest/Dockerfile)
+
+# Caution
+
+This image is built per commit.
+If you want to use the latest docker image, you should remove the existing image, and pull it once again.
+
+# What is vulsrepo?
+
+VulsRepo is visualized based on the json report output in [vuls](https://github.com/future-architect/vuls).
+
+# How to use this image
+
+## vulsrepo
+
+```console
+$docker run -dt \
+    -v $PWD:/vuls \
+    -p 80:80 \
+    vuls/vulsrepo
+```
+
+# User Feedback
+
+## Documentation
+
+Documentation for this image is stored in the [`docker/` directory](https://github.com/future-architect/vuls/tree/master/setup/docker) of the [`future-architect/vuls` GitHub repo](https://github.com/future-architect/vuls). 
+
+## Issues
+
+If you have any problems with or questions about this image, please contact us through a [GitHub issue](https://github.com/future-architect/vuls/issues). 
+
+## Contributing
+
+1. fork a repository: github.com/future-architect/vuls to github.com/you/repo
+1. get original code: go get github.com/future-architect/vuls
+1. work on original code
+1. add remote to your repo: git remote add myfork https://github.com/you/repo.git
+1. push your changes: git push myfork
+1. create a new Pull Request

util/logutil.go

@@ -18,8 +18,9 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 package util
 
 import (
-	"fmt"
 	"os"
+	"path/filepath"
+	"runtime"
 
 	"github.com/Sirupsen/logrus"
 	"github.com/rifflock/lfshook"
@@ -40,6 +41,9 @@ func NewCustomLogger(c config.ServerInfo) *logrus.Entry {
 
 	// File output
 	logDir := "/var/log/vuls"
+	if runtime.GOOS == "windows" {
+		logDir = filepath.Join(os.Getenv("APPDATA"), "vuls")
+	}
 	if _, err := os.Stat(logDir); os.IsNotExist(err) {
 		if err := os.Mkdir(logDir, 0666); err != nil {
 			logrus.Errorf("Failed to create log directory: %s", err)
@@ -48,11 +52,11 @@ func NewCustomLogger(c config.ServerInfo) *logrus.Entry {
 
 	whereami := "localhost"
 	if 0 < len(c.ServerName) {
-		whereami = fmt.Sprintf("%s:%s", c.ServerName, c.Port)
-
+		whereami = c.GetServerName()
 	}
+
 	if _, err := os.Stat(logDir); err == nil {
-		path := fmt.Sprintf("%s/%s.log", logDir, whereami)
+		path := filepath.Join(logDir, whereami)
 		log.Hooks.Add(lfshook.NewHook(lfshook.PathMap{
 			logrus.DebugLevel: path,
 			logrus.InfoLevel:  path,

util/util.go

@@ -31,6 +31,12 @@ func GenWorkers(num int) chan<- func() {
 	tasks := make(chan func())
 	for i := 0; i < num; i++ {
 		go func() {
+			defer func() {
+				if p := recover(); p != nil {
+					log := NewCustomLogger(config.ServerInfo{})
+					log.Debugf("Panic: %s")
+				}
+			}()
 			for f := range tasks {
 				f()
 			}
@@ -105,7 +111,7 @@ func ProxyEnv() string {
 
 // PrependProxyEnv prepends proxy enviroment variable
 func PrependProxyEnv(cmd string) string {
-	if config.Conf.HTTPProxy == "" {
+	if len(config.Conf.HTTPProxy) == 0 {
 		return cmd
 	}
 	return fmt.Sprintf("%s %s", ProxyEnv(), cmd)
@@ -118,3 +124,14 @@ func PrependProxyEnv(cmd string) string {
 //      }
 //      return time.Unix(i, 0), nil
 //  }
+
+// Truncate truncates string to the length
+func Truncate(str string, length int) string {
+	if length < 0 {
+		return str
+	}
+	if length <= len(str) {
+		return str[:length]
+	}
+	return str
+}

util/util_test.go

@@ -131,3 +131,43 @@ func TestPrependHTTPProxyEnv(t *testing.T) {
 	}
 
 }
+
+func TestTruncate(t *testing.T) {
+	var tests = []struct {
+		in     string
+		length int
+		out    string
+	}{
+		{
+			in:     "abcde",
+			length: 3,
+			out:    "abc",
+		},
+		{
+			in:     "abcdefg",
+			length: 5,
+			out:    "abcde",
+		},
+		{
+			in:     "abcdefg",
+			length: 10,
+			out:    "abcdefg",
+		},
+		{
+			in:     "abcdefg",
+			length: 0,
+			out:    "",
+		},
+		{
+			in:     "abcdefg",
+			length: -1,
+			out:    "abcdefg",
+		},
+	}
+	for _, tt := range tests {
+		actual := Truncate(tt.in, tt.length)
+		if actual != tt.out {
+			t.Errorf("\nexpected: %s\n  actual: %s", tt.out, actual)
+		}
+	}
+}

version.go

@@ -1,24 +0,0 @@
-/* Vuls - Vulnerability Scanner
-Copyright (C) 2016  Future Architect, Inc. Japan.
-
-This program is free software: you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation, either version 3 of the License, or
-(at your option) any later version.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program.  If not, see <http://www.gnu.org/licenses/>.
-*/
-
-package main
-
-// Name is Vuls
-const Name string = "vuls"
-
-// Version of Vuls
-const Version string = "0.1.3"