feat(contrib/fvuls) Add commands to obtained CPE information of network devices by executing snmp2cpe and upload to Fvuls server (#1721)

* add: README.md

* add: commands(discover,add-server,add-cpe)

* add: implements(discover,add-server,add-cpe)

* fix: changed os.Exit(1) in main.go to return an error

* fix: lint error

* delete: trivy-to-vuls stdIn

* fix: Incomprehesible error logs

* fix: according to review

* add: function converts old config to latest one

* delete: add-server

* fix: lint error

* fix

* fix: remote scan error in Windows

* fix: lint error

* fix

* fix: lint error

* fix: lint error

* fix: lint error

* add: scanner/scanner.go test normalizeHomeDirForWindows()

* fix

* fix

* fix

* fix

* fix

* fix

* fix: lint error

* fix: error log

* fix

* refactor(fvuls)

* Refactor (#2)

refactor
---------

Co-authored-by: 和田皓翔 <wadahiroka@192.168.0.6>

* Refactor (#3)

 fix

---------

Co-authored-by: Sadayuki Matsuno <sadayuki.matsuno@gmail.com>
Co-authored-by: 和田皓翔 <wadahiroka@192.168.0.6>

* fix

* fix: lint error

* fix

---------

Co-authored-by: 和田皓翔 <wadahiroka@192.168.0.4>
Co-authored-by: 和田皓翔 <wadahiroka@192.168.0.10>
Co-authored-by: 和田皓翔 <wadahiroka@192.168.0.6>
Co-authored-by: Sadayuki Matsuno <sadayuki.matsuno@gmail.com>
This commit is contained in:
hiroka-wada
2023-09-21 15:55:05 +09:00
committed by GitHub
parent 3f2dbe3b6d
commit 80b48fcbaa
11 changed files with 815 additions and 61 deletions

8
.gitignore vendored
View File

@@ -3,6 +3,7 @@
*.swp
*.sqlite3*
*.db
*.toml
tags
.gitmodules
coverage.out
@@ -10,7 +11,6 @@ issues/
vendor/
log/
results
config.toml
!setup/docker/*
.DS_Store
dist/
@@ -18,7 +18,7 @@ dist/
vuls.*
vuls
!cmd/vuls
future-vuls
trivy-to-vuls
/future-vuls
/trivy-to-vuls
snmp2cpe
!snmp2cpe/
!snmp2cpe/

View File

@@ -2,18 +2,77 @@
## Main Features
- upload vuls results json to future-vuls
- `future-vuls upload`
- upload vuls results json to future-vuls
- `future-vuls discover`
- Explore hosts within the CIDR range using the ping command
- Describe the information including CPE on the found hosts in a toml-formatted file.
- Exec snmp2cpe(https://github.com/future-architect/vuls/pull/1625) to active hosts to obtain CPE<br>
Commands running internally  `snmp2cpe v2c {IPAddr} public | snmp2cpe convert`<br>
Structure of toml-formatted file
```
[server.{ip}]
ip = {IpAddr}
server_name = ""
uuid = {UUID}
cpe_uris = []
fvuls_sync = false
```
- `future-vuls add-cpe`
- Create pseudo server to Fvuls to obtain uuid and Upload CPE information on the specified(FvulsSync is true and UUID is obtained) hosts to Fvuls
- Fvuls_Sync must be rewritten to true to designate it as the target of the command<br><br>
1. `future-vuls discover`
2. `future-vuls add-cpe`
These two commands are used to manage the CPE of network devices, and by executing the commands in the order from the top, you can manage the CPE of each device in Fvuls
toml file after command execution
```
["192.168.0.10"]
ip = "192.168.0.10"
server_name = "192.168.0.10"
uuid = "e811e2b1-9463-d682-7c79-a4ab37de28cf"
cpe_uris = ["cpe:2.3:h:fortinet:fortigate-50e:-:*:*:*:*:*:*:*", "cpe:2.3:o:fortinet:fortios:5.4.6:*:*:*:*:*:*:*"]
fvuls_sync = true
```
## Installation
```
git clone https://github.com/future-architect/vuls.git
cd vuls
make build-future-vuls
```
## Command Reference
```
./future-vuls -h
Usage:
future-vuls [command]
Available Commands:
add-cpe Create a pseudo server in Fvuls and register CPE. Default outputFile is ./discover_list.toml
completion Generate the autocompletion script for the specified shell
discover discover hosts with CIDR range. Run snmp2cpe on active host to get CPE. Default outputFile is ./discover_list.toml
help Help about any command
upload Upload to FutureVuls
version Show version
Flags:
-h, --help help for future-vuls
Use "future-vuls [command] --help" for more information about a command.
```
### Subcommands
```
./future-vuls upload -h
Upload to FutureVuls
Usage:
@@ -29,10 +88,71 @@ Flags:
--uuid string server uuid. ENV: VULS_SERVER_UUID
```
```
./future-vuls discover -h
discover hosts with CIDR range. Run snmp2cpe on active host to get CPE. Default outputFile is ./discover_list.toml
Usage:
future-vuls discover --cidr <CIDR_RANGE> --output <OUTPUT_FILE> [flags]
Examples:
future-vuls discover --cidr 192.168.0.0/24 --output discover_list.toml
Flags:
--cidr string cidr range
-h, --help help for discover
--output string output file
--snmp-version string snmp version v1,v2c and v3. default: v2c
```
```
./future-vuls add-cpe -h
Create a pseudo server in Fvuls and register CPE. Default outputFile is ./discover_list.toml
Usage:
future-vuls add-cpe --token <VULS_TOKEN> --output <OUTPUT_FILE> [flags]
Examples:
future-vuls add-cpe --token <VULS_TOKEN>
Flags:
-h, --help help for add-cpe
--http-proxy string proxy url
--output string output file
-t, --token string future vuls token ENV: VULS_TOKEN
```
## Usage
- update results json
- `future-vuls upload`
```
cat results.json | future-vuls upload --stdin --token xxxx --url https://xxxx --group-id 1 --uuid xxxx
```
```
- `future-vuls discover`
```
./future-vuls discover --cidr 192.168.0.1/24
Discovering 192.168.0.1/24...
192.168.0.1: Execute snmp2cpe...
failed to execute snmp2cpe. err: failed to execute snmp2cpe. err: exit status 1
192.168.0.2: Execute snmp2cpe...
failed to execute snmp2cpe. err: failed to execute snmp2cpe. err: exit status 1
192.168.0.4: Execute snmp2cpe...
failed to execute snmp2cpe. err: failed to execute snmp2cpe. err: exit status 1
192.168.0.5: Execute snmp2cpe...
failed to execute snmp2cpe. err: failed to execute snmp2cpe. err: exit status 1
192.168.0.6: Execute snmp2cpe...
New network device found 192.168.0.6
wrote to discover_list.toml
```
- `future-vuls add-cpe`
```
./future-vuls add-cpe --token fvgr-686b92af-5216-11ee-a241-0a58a9feac02
Creating 1 pseudo server...
192.168.0.6: Created FutureVuls pseudo server ce024b45-1c59-5b86-1a67-e78a40dfec01
wrote to discover_list.toml
Uploading 1 server's CPE...
192.168.0.6: Uploaded CPE cpe:2.3:h:fortinet:fortigate-50e:-:*:*:*:*:*:*:*
192.168.0.6: Uploaded CPE cpe:2.3:o:fortinet:fortios:5.4.6:*:*:*:*:*:*:*
```

View File

@@ -1,118 +1,162 @@
// Package main ...
package main
import (
"bufio"
"bytes"
"encoding/json"
"fmt"
"os"
"strconv"
"strings"
"github.com/future-architect/vuls/config"
"github.com/future-architect/vuls/models"
"github.com/future-architect/vuls/saas"
cidrPkg "github.com/3th1nk/cidr"
vulsConfig "github.com/future-architect/vuls/config"
"github.com/future-architect/vuls/contrib/future-vuls/pkg/config"
"github.com/future-architect/vuls/contrib/future-vuls/pkg/cpe"
"github.com/future-architect/vuls/contrib/future-vuls/pkg/discover"
"github.com/future-architect/vuls/contrib/future-vuls/pkg/fvuls"
"github.com/spf13/cobra"
)
var (
configFile string
stdIn bool
jsonDir string
serverUUID string
groupID int64
token string
tags []string
url string
configFile string
stdIn bool
jsonDir string
serverUUID string
groupID int64
token string
tags []string
outputFile string
cidr string
snmpVersion string
proxy string
)
func main() {
var err error
var cmdVersion = &cobra.Command{
Use: "version",
Short: "Show version",
Long: "Show version",
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("future-vuls-%s-%s\n", vulsConfig.Version, vulsConfig.Revision)
},
}
var cmdFvulsUploader = &cobra.Command{
Use: "upload",
Short: "Upload to FutureVuls",
Long: `Upload to FutureVuls`,
Run: func(cmd *cobra.Command, args []string) {
RunE: func(cmd *cobra.Command, args []string) error {
if len(serverUUID) == 0 {
serverUUID = os.Getenv("VULS_SERVER_UUID")
}
if groupID == 0 {
envGroupID := os.Getenv("VULS_GROUP_ID")
if groupID, err = strconv.ParseInt(envGroupID, 10, 64); err != nil {
fmt.Printf("Invalid GroupID: %s\n", envGroupID)
return
return fmt.Errorf("invalid GroupID: %s", envGroupID)
}
}
if len(url) == 0 {
url = os.Getenv("VULS_URL")
}
if len(token) == 0 {
token = os.Getenv("VULS_TOKEN")
}
if len(tags) == 0 {
tags = strings.Split(os.Getenv("VULS_TAGS"), ",")
}
var scanResultJSON []byte
if stdIn {
reader := bufio.NewReader(os.Stdin)
buf := new(bytes.Buffer)
if _, err = buf.ReadFrom(reader); err != nil {
return
if _, err := buf.ReadFrom(reader); err != nil {
return fmt.Errorf("failed to read from stdIn. err: %v", err)
}
scanResultJSON = buf.Bytes()
} else {
fmt.Println("use --stdin option")
os.Exit(1)
return
return fmt.Errorf("use --stdin option")
}
fvulsClient := fvuls.NewClient(token, "")
if err := fvulsClient.UploadToFvuls(serverUUID, groupID, tags, scanResultJSON); err != nil {
fmt.Printf("%v", err)
// avoid to display help message
os.Exit(1)
}
return nil
},
}
var scanResult models.ScanResult
if err = json.Unmarshal(scanResultJSON, &scanResult); err != nil {
fmt.Println("Failed to parse json", err)
os.Exit(1)
return
var cmdDiscover = &cobra.Command{
Use: "discover --cidr <CIDR_RANGE> --output <OUTPUT_FILE>",
Short: "discover hosts with CIDR range. Run snmp2cpe on active host to get CPE. Default outputFile is ./discover_list.toml",
Example: "future-vuls discover --cidr 192.168.0.0/24 --output discover_list.toml",
RunE: func(cmd *cobra.Command, args []string) error {
if len(outputFile) == 0 {
outputFile = config.DiscoverTomlFileName
}
scanResult.ServerUUID = serverUUID
if 0 < len(tags) {
if scanResult.Optional == nil {
scanResult.Optional = map[string]interface{}{}
if len(cidr) == 0 {
return fmt.Errorf("please specify cidr range")
}
if _, err := cidrPkg.Parse(cidr); err != nil {
return fmt.Errorf("Invalid cidr range")
}
if len(snmpVersion) == 0 {
snmpVersion = config.SnmpVersion
}
if snmpVersion != "v1" && snmpVersion != "v2c" && snmpVersion != "v3" {
return fmt.Errorf("Invalid snmpVersion")
}
if err := discover.ActiveHosts(cidr, outputFile, snmpVersion); err != nil {
fmt.Printf("%v", err)
// avoid to display help message
os.Exit(1)
}
return nil
},
}
var cmdAddCpe = &cobra.Command{
Use: "add-cpe --token <VULS_TOKEN> --output <OUTPUT_FILE>",
Short: "Create a pseudo server in Fvuls and register CPE. Default outputFile is ./discover_list.toml",
Example: "future-vuls add-cpe --token <VULS_TOKEN>",
RunE: func(cmd *cobra.Command, args []string) error {
if len(token) == 0 {
token = os.Getenv("VULS_TOKEN")
if len(token) == 0 {
return fmt.Errorf("token not specified")
}
scanResult.Optional["VULS_TAGS"] = tags
}
config.Conf.Saas.GroupID = groupID
config.Conf.Saas.Token = token
config.Conf.Saas.URL = url
if err = (saas.Writer{}).Write(scanResult); err != nil {
fmt.Println(err)
if len(outputFile) == 0 {
outputFile = config.DiscoverTomlFileName
}
if err := cpe.AddCpe(token, outputFile, proxy); err != nil {
fmt.Printf("%v", err)
// avoid to display help message
os.Exit(1)
return
}
return
},
}
var cmdVersion = &cobra.Command{
Use: "version",
Short: "Show version",
Long: "Show version",
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("future-vuls-%s-%s\n", config.Version, config.Revision)
return nil
},
}
cmdFvulsUploader.PersistentFlags().StringVar(&serverUUID, "uuid", "", "server uuid. ENV: VULS_SERVER_UUID")
cmdFvulsUploader.PersistentFlags().StringVar(&configFile, "config", "", "config file (default is $HOME/.cobra.yaml)")
cmdFvulsUploader.PersistentFlags().BoolVarP(&stdIn, "stdin", "s", false, "input from stdin. ENV: VULS_STDIN")
// TODO Read JSON file from directory
// cmdFvulsUploader.Flags().StringVarP(&jsonDir, "results-dir", "d", "./", "vuls scan results json dir")
cmdFvulsUploader.PersistentFlags().Int64VarP(&groupID, "group-id", "g", 0, "future vuls group id, ENV: VULS_GROUP_ID")
cmdFvulsUploader.PersistentFlags().StringVarP(&token, "token", "t", "", "future vuls token")
cmdFvulsUploader.PersistentFlags().StringVar(&url, "url", "", "future vuls upload url")
cmdDiscover.PersistentFlags().StringVar(&cidr, "cidr", "", "cidr range")
cmdDiscover.PersistentFlags().StringVar(&outputFile, "output", "", "output file")
cmdDiscover.PersistentFlags().StringVar(&snmpVersion, "snmp-version", "", "snmp version v1,v2c and v3. default: v2c ")
cmdAddCpe.PersistentFlags().StringVarP(&token, "token", "t", "", "future vuls token ENV: VULS_TOKEN")
cmdAddCpe.PersistentFlags().StringVar(&outputFile, "output", "", "output file")
cmdAddCpe.PersistentFlags().StringVar(&proxy, "http-proxy", "", "proxy url")
var rootCmd = &cobra.Command{Use: "future-vuls"}
rootCmd.AddCommand(cmdDiscover)
rootCmd.AddCommand(cmdAddCpe)
rootCmd.AddCommand(cmdFvulsUploader)
rootCmd.AddCommand(cmdVersion)
if err = rootCmd.Execute(); err != nil {
fmt.Println("Failed to execute command", err)
fmt.Println("Failed to execute command")
}
}

View File

@@ -0,0 +1,23 @@
// Package config ...
package config
const (
DiscoverTomlFileName = "discover_list.toml"
SnmpVersion = "v2c"
FvulsDomain = "vuls.biz"
DiscoverTomlTimeStampFormat = "20060102150405"
)
// DiscoverToml ...
type DiscoverToml map[string]ServerSetting
// ServerSetting ...
type ServerSetting struct {
IP string `toml:"ip"`
ServerName string `toml:"server_name"`
UUID string `toml:"uuid"`
CpeURIs []string `toml:"cpe_uris"`
FvulsSync bool `toml:"fvuls_sync"`
// use internal
NewCpeURIs []string `toml:"-"`
}

View File

@@ -0,0 +1,186 @@
// Package cpe ...
package cpe
import (
"context"
"fmt"
"os"
"time"
"github.com/BurntSushi/toml"
"github.com/future-architect/vuls/contrib/future-vuls/pkg/config"
"github.com/future-architect/vuls/contrib/future-vuls/pkg/fvuls"
"golang.org/x/exp/slices"
)
// AddCpeConfig ...
type AddCpeConfig struct {
Token string
Proxy string
DiscoverTomlPath string
OriginalDiscoverToml config.DiscoverToml
}
// AddCpe ...
func AddCpe(token, outputFile, proxy string) (err error) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
defer cancel()
cpeConfig := &AddCpeConfig{
Token: token,
Proxy: proxy,
DiscoverTomlPath: outputFile,
}
var needAddServers, needAddCpes config.DiscoverToml
if needAddServers, needAddCpes, err = cpeConfig.LoadAndCheckTomlFile(ctx); err != nil {
return err
}
if 0 < len(needAddServers) {
addedServers := cpeConfig.AddServerToFvuls(ctx, needAddServers)
if 0 < len(addedServers) {
for name, server := range addedServers {
needAddCpes[name] = server
}
}
// update discover toml
for name, server := range needAddCpes {
cpeConfig.OriginalDiscoverToml[name] = server
}
if err = cpeConfig.WriteDiscoverToml(); err != nil {
return err
}
}
if 0 < len(needAddCpes) {
var addedCpes config.DiscoverToml
if addedCpes, err = cpeConfig.AddCpeToFvuls(ctx, needAddCpes); err != nil {
return err
}
for name, server := range addedCpes {
cpeConfig.OriginalDiscoverToml[name] = server
}
if err = cpeConfig.WriteDiscoverToml(); err != nil {
return err
}
}
return nil
}
// LoadAndCheckTomlFile ...
func (c *AddCpeConfig) LoadAndCheckTomlFile(ctx context.Context) (needAddServers, needAddCpes config.DiscoverToml, err error) {
var discoverToml config.DiscoverToml
if _, err = toml.DecodeFile(c.DiscoverTomlPath, &discoverToml); err != nil {
return nil, nil, fmt.Errorf("failed to read discover toml: %s", c.DiscoverTomlPath)
}
c.OriginalDiscoverToml = discoverToml
needAddServers = make(map[string]config.ServerSetting)
needAddCpes = make(map[string]config.ServerSetting)
for name, setting := range discoverToml {
if !setting.FvulsSync {
continue
}
if setting.UUID == "" {
setting.NewCpeURIs = setting.CpeURIs
needAddServers[name] = setting
} else if 0 < len(setting.CpeURIs) {
fvulsClient := fvuls.NewClient(c.Token, c.Proxy)
var serverDetail fvuls.ServerDetailOutput
if serverDetail, err = fvulsClient.GetServerByUUID(ctx, setting.UUID); err != nil {
fmt.Printf("%s: Failed to Fetch serverID. err: %v\n", name, err)
continue
}
// update server name
server := c.OriginalDiscoverToml[name]
server.ServerName = serverDetail.ServerName
c.OriginalDiscoverToml[name] = server
var uploadedCpes []string
if uploadedCpes, err = fvulsClient.ListUploadedCPE(ctx, serverDetail.ServerID); err != nil {
fmt.Printf("%s: Failed to Fetch uploaded CPE. err: %v\n", name, err)
continue
}
// check if there are any CPEs that are not uploaded
var newCpes []string
for _, cpeURI := range setting.CpeURIs {
if !slices.Contains(uploadedCpes, cpeURI) {
newCpes = append(newCpes, cpeURI)
}
}
if 0 < len(newCpes) {
setting.NewCpeURIs = newCpes
needAddCpes[name] = setting
}
}
}
if len(needAddServers)+len(needAddCpes) == 0 {
fmt.Printf("There are no hosts to add to Fvuls\n")
return nil, nil, nil
}
return needAddServers, needAddCpes, nil
}
// AddServerToFvuls ...
func (c *AddCpeConfig) AddServerToFvuls(ctx context.Context, needAddServers map[string]config.ServerSetting) (addedServers config.DiscoverToml) {
fmt.Printf("Creating %d pseudo server...\n", len(needAddServers))
fvulsClient := fvuls.NewClient(c.Token, c.Proxy)
addedServers = make(map[string]config.ServerSetting)
for name, server := range needAddServers {
var serverDetail fvuls.ServerDetailOutput
serverDetail, err := fvulsClient.CreatePseudoServer(ctx, server.ServerName)
if err != nil {
fmt.Printf("%s: Failed to add to Fvuls server. err: %v\n", server.ServerName, err)
continue
}
server.UUID = serverDetail.ServerUUID
server.ServerName = serverDetail.ServerName
addedServers[name] = server
fmt.Printf("%s: Created FutureVuls pseudo server %s\n", server.ServerName, server.UUID)
}
return addedServers
}
// AddCpeToFvuls ...
func (c *AddCpeConfig) AddCpeToFvuls(ctx context.Context, needAddCpes config.DiscoverToml) (config.DiscoverToml, error) {
fmt.Printf("Uploading %d server's CPE...\n", len(needAddCpes))
fvulsClient := fvuls.NewClient(c.Token, c.Proxy)
for name, server := range needAddCpes {
serverDetail, err := fvulsClient.GetServerByUUID(ctx, server.UUID)
server.ServerName = serverDetail.ServerName
if err != nil {
fmt.Printf("%s: Failed to Fetch serverID. err: %v\n", server.ServerName, err)
continue
}
for _, cpeURI := range server.NewCpeURIs {
if err = fvulsClient.UploadCPE(ctx, cpeURI, serverDetail.ServerID); err != nil {
fmt.Printf("%s: Failed to upload CPE %s. err: %v\n", server.ServerName, cpeURI, err)
continue
}
fmt.Printf("%s: Uploaded CPE %s\n", server.ServerName, cpeURI)
}
needAddCpes[name] = server
}
return needAddCpes, nil
}
// WriteDiscoverToml ...
func (c *AddCpeConfig) WriteDiscoverToml() error {
f, err := os.OpenFile(c.DiscoverTomlPath, os.O_RDWR, 0666)
if err != nil {
return fmt.Errorf("failed to open toml file. err: %v", err)
}
defer f.Close()
encoder := toml.NewEncoder(f)
if err := encoder.Encode(c.OriginalDiscoverToml); err != nil {
return fmt.Errorf("failed to write to %s. err: %v", c.DiscoverTomlPath, err)
}
fmt.Printf("wrote to %s\n\n", c.DiscoverTomlPath)
return nil
}

View File

@@ -0,0 +1,127 @@
// Package discover ...
package discover
import (
"encoding/json"
"fmt"
"io"
"os"
"os/exec"
"time"
"github.com/BurntSushi/toml"
"github.com/future-architect/vuls/contrib/future-vuls/pkg/config"
"github.com/kotakanbe/go-pingscanner"
)
// ActiveHosts ...
func ActiveHosts(cidr string, outputFile string, snmpVersion string) error {
scanner := pingscanner.PingScanner{
CIDR: cidr,
PingOptions: []string{
"-c1",
},
NumOfConcurrency: 100,
}
fmt.Printf("Discovering %s...\n", cidr)
activeHosts, err := scanner.Scan()
if err != nil {
return fmt.Errorf("host Discovery failed. err: %v", err)
}
if len(activeHosts) == 0 {
return fmt.Errorf("active hosts not found in %s", cidr)
}
discoverToml := config.DiscoverToml{}
if _, err := os.Stat(outputFile); err == nil {
fmt.Printf("%s is found.\n", outputFile)
if _, err = toml.DecodeFile(outputFile, &discoverToml); err != nil {
return fmt.Errorf("failed to read discover toml: %s", outputFile)
}
}
servers := make(config.DiscoverToml)
for _, activeHost := range activeHosts {
cpes, err := executeSnmp2cpe(activeHost, snmpVersion)
if err != nil {
fmt.Printf("failed to execute snmp2cpe. err: %v\n", err)
continue
}
fvulsSync := false
serverUUID := ""
serverName := activeHost
if server, ok := discoverToml[activeHost]; ok {
fvulsSync = server.FvulsSync
serverUUID = server.UUID
serverName = server.ServerName
} else {
fmt.Printf("New network device found %s\n", activeHost)
}
servers[activeHost] = config.ServerSetting{
IP: activeHost,
ServerName: serverName,
UUID: serverUUID,
FvulsSync: fvulsSync,
CpeURIs: cpes[activeHost],
}
}
for ip, setting := range discoverToml {
if _, ok := servers[ip]; !ok {
fmt.Printf("%s(%s) has been removed as there was no response.\n", setting.ServerName, setting.IP)
}
}
if len(servers) == 0 {
return fmt.Errorf("new network devices could not be found")
}
if 0 < len(discoverToml) {
fmt.Printf("Creating new %s and saving the old file under different name...\n", outputFile)
timestamp := time.Now().Format(config.DiscoverTomlTimeStampFormat)
oldDiscoverFile := fmt.Sprintf("%s_%s", timestamp, outputFile)
if err := os.Rename(outputFile, oldDiscoverFile); err != nil {
return fmt.Errorf("failed to rename exist toml file. err: %v", err)
}
fmt.Printf("You can check the difference from the previous DISCOVER with the following command.\n diff %s %s\n", outputFile, oldDiscoverFile)
}
f, err := os.OpenFile(outputFile, os.O_RDWR|os.O_CREATE, 0666)
if err != nil {
return fmt.Errorf("failed to open toml file. err: %v", err)
}
defer f.Close()
encoder := toml.NewEncoder(f)
if err = encoder.Encode(servers); err != nil {
return fmt.Errorf("failed to write to %s. err: %v", outputFile, err)
}
fmt.Printf("wrote to %s\n", outputFile)
return nil
}
func executeSnmp2cpe(addr string, snmpVersion string) (cpes map[string][]string, err error) {
fmt.Printf("%s: Execute snmp2cpe...\n", addr)
result, err := exec.Command("./snmp2cpe", snmpVersion, addr, "public").CombinedOutput()
if err != nil {
return nil, fmt.Errorf("failed to execute snmp2cpe. err: %v", err)
}
cmd := exec.Command("./snmp2cpe", "convert")
stdin, err := cmd.StdinPipe()
if err != nil {
return nil, fmt.Errorf("failed to convert snmp2cpe result. err: %v", err)
}
if _, err := io.WriteString(stdin, string(result)); err != nil {
return nil, fmt.Errorf("failed to write to stdIn. err: %v", err)
}
stdin.Close()
output, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("failed to convert snmp2cpe result. err: %v", err)
}
if err := json.Unmarshal(output, &cpes); err != nil {
return nil, fmt.Errorf("failed to unmarshal snmp2cpe output. err: %v", err)
}
return cpes, nil
}

View File

@@ -0,0 +1,192 @@
// Package fvuls ...
package fvuls
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"github.com/future-architect/vuls/config"
"github.com/future-architect/vuls/models"
"github.com/future-architect/vuls/saas"
"github.com/future-architect/vuls/util"
)
// Client ...
type Client struct {
Token string
Proxy string
FvulsScanEndpoint string
FvulsRestEndpoint string
}
// NewClient ...
func NewClient(token string, proxy string) *Client {
fvulsDomain := "vuls.biz"
if domain := os.Getenv("VULS_DOMAIN"); 0 < len(domain) {
fvulsDomain = domain
}
return &Client{
Token: token,
Proxy: proxy,
FvulsScanEndpoint: fmt.Sprintf("https://auth.%s/one-time-auth", fvulsDomain),
FvulsRestEndpoint: fmt.Sprintf("https://rest.%s/v1", fvulsDomain),
}
}
// UploadToFvuls ...
func (f Client) UploadToFvuls(serverUUID string, groupID int64, tags []string, scanResultJSON []byte) error {
var scanResult models.ScanResult
if err := json.Unmarshal(scanResultJSON, &scanResult); err != nil {
fmt.Printf("failed to parse json. err: %v\nPerhaps scan has failed. Please check following scan results.\nResult: %s", err, fmt.Sprintf("%s", scanResultJSON))
return err
}
scanResult.ServerUUID = serverUUID
if 0 < len(tags) {
if scanResult.Optional == nil {
scanResult.Optional = map[string]interface{}{}
}
scanResult.Optional["VULS_TAGS"] = tags
}
config.Conf.Saas.GroupID = groupID
config.Conf.Saas.Token = f.Token
config.Conf.Saas.URL = f.FvulsScanEndpoint
if err := (saas.Writer{}).Write(scanResult); err != nil {
return fmt.Errorf("%v", err)
}
return nil
}
// GetServerByUUID ...
func (f Client) GetServerByUUID(ctx context.Context, uuid string) (server ServerDetailOutput, err error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, fmt.Sprintf("%s/server/uuid/%s", f.FvulsRestEndpoint, uuid), nil)
if err != nil {
return ServerDetailOutput{}, fmt.Errorf("failed to create request. err: %v", err)
}
t, err := f.sendHTTPRequest(req)
if err != nil {
return ServerDetailOutput{}, err
}
var serverDetail ServerDetailOutput
if err := json.Unmarshal(t, &serverDetail); err != nil {
if err.Error() == "invalid character 'A' looking for beginning of value" {
return ServerDetailOutput{}, fmt.Errorf("invalid token")
}
return ServerDetailOutput{}, fmt.Errorf("failed to unmarshal serverDetail. err: %v", err)
}
return serverDetail, nil
}
// CreatePseudoServer ...
func (f Client) CreatePseudoServer(ctx context.Context, name string) (serverDetail ServerDetailOutput, err error) {
payload := CreatePseudoServerInput{
ServerName: name,
}
body, err := json.Marshal(payload)
if err != nil {
return ServerDetailOutput{}, fmt.Errorf("failed to Marshal to JSON: %v", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, fmt.Sprintf("%s/server/pseudo", f.FvulsRestEndpoint), bytes.NewBuffer(body))
if err != nil {
return ServerDetailOutput{}, fmt.Errorf("failed to create request: %v", err)
}
t, err := f.sendHTTPRequest(req)
if err != nil {
return ServerDetailOutput{}, err
}
if err := json.Unmarshal(t, &serverDetail); err != nil {
if err.Error() == "invalid character 'A' looking for beginning of value" {
return ServerDetailOutput{}, fmt.Errorf("invalid token")
}
return ServerDetailOutput{}, fmt.Errorf("failed to unmarshal serverDetail. err: %v", err)
}
return serverDetail, nil
}
// UploadCPE ...
func (f Client) UploadCPE(ctx context.Context, cpeURI string, serverID int64) (err error) {
payload := AddCpeInput{
ServerID: serverID,
CpeName: cpeURI,
IsURI: false,
}
body, err := json.Marshal(payload)
if err != nil {
return fmt.Errorf("failed to marshal JSON: %v", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, fmt.Sprintf("%s/pkgCpe/cpe", f.FvulsRestEndpoint), bytes.NewBuffer(body))
if err != nil {
return fmt.Errorf("failed to create request. err: %v", err)
}
t, err := f.sendHTTPRequest(req)
if err != nil {
return err
}
var cpeDetail AddCpeOutput
if err := json.Unmarshal(t, &cpeDetail); err != nil {
if err.Error() == "invalid character 'A' looking for beginning of value" {
return fmt.Errorf("invalid token")
}
return fmt.Errorf("failed to unmarshal serverDetail. err: %v", err)
}
return nil
}
// ListUploadedCPE ...
func (f Client) ListUploadedCPE(ctx context.Context, serverID int64) (uploadedCPEs []string, err error) {
page := 1
for {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, fmt.Sprintf("%s/pkgCpes?page=%d&limit=%d&filterServerID=%d", f.FvulsRestEndpoint, page, 200, serverID), nil)
if err != nil {
return nil, fmt.Errorf("failed to create request. err: %v", err)
}
t, err := f.sendHTTPRequest(req)
if err != nil {
return nil, err
}
var pkgCpes ListCpesOutput
if err := json.Unmarshal(t, &pkgCpes); err != nil {
if err.Error() == "invalid character 'A' looking for beginning of value" {
return nil, fmt.Errorf("invalid token")
}
return nil, fmt.Errorf("failed to unmarshal listCpesOutput. err: %v", err)
}
for _, pkgCpe := range pkgCpes.PkgCpes {
uploadedCPEs = append(uploadedCPEs, pkgCpe.CpeFS)
}
if pkgCpes.Paging.TotalPage <= page {
break
}
page++
}
return uploadedCPEs, nil
}
func (f Client) sendHTTPRequest(req *http.Request) ([]byte, error) {
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
req.Header.Set("Authorization", f.Token)
client, err := util.GetHTTPClient(f.Proxy)
if err != nil {
return nil, fmt.Errorf("%v", err)
}
resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("failed to sent request. err: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return nil, fmt.Errorf("error response: %v", resp.StatusCode)
}
t, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("failed to read response data. err: %v", err)
}
return t, nil
}

View File

@@ -0,0 +1,56 @@
// Package fvuls ...
package fvuls
// CreatePseudoServerInput ...
type CreatePseudoServerInput struct {
ServerName string `json:"serverName"`
}
// AddCpeInput ...
type AddCpeInput struct {
ServerID int64 `json:"serverID"`
CpeName string `json:"cpeName"`
IsURI bool `json:"isURI"`
}
// AddCpeOutput ...
type AddCpeOutput struct {
Server ServerChild `json:"server"`
}
// ListCpesInput ...
type ListCpesInput struct {
Page int `json:"page"`
Limit int `json:"limit"`
ServerID int64 `json:"filterServerID"`
}
// ListCpesOutput ...
type ListCpesOutput struct {
Paging Paging `json:"paging"`
PkgCpes []PkgCpes `json:"pkgCpes"`
}
// Paging ...
type Paging struct {
Page int `json:"page"`
Limit int `json:"limit"`
TotalPage int `json:"totalPage"`
}
// PkgCpes ...
type PkgCpes struct {
CpeFS string `json:"cpeFS"`
}
// ServerChild ...
type ServerChild struct {
ServerName string `json:"serverName"`
}
// ServerDetailOutput ...
type ServerDetailOutput struct {
ServerID int64 `json:"id"`
ServerName string `json:"serverName"`
ServerUUID string `json:"serverUuid"`
}

View File

@@ -2,6 +2,7 @@ package parser
import (
"encoding/json"
"fmt"
v2 "github.com/future-architect/vuls/contrib/trivy/parser/v2"
"github.com/future-architect/vuls/models"
@@ -22,6 +23,7 @@ type Report struct {
func NewParser(vulnJSON []byte) (Parser, error) {
r := Report{}
if err := json.Unmarshal(vulnJSON, &r); err != nil {
fmt.Printf("%s", fmt.Sprintf("%s", vulnJSON))
return nil, xerrors.Errorf("Failed to parse JSON. Please use the latest version of trivy, trivy-to-vuls and future-vuls")
}
switch r.SchemaVersion {

1
go.mod
View File

@@ -3,6 +3,7 @@ module github.com/future-architect/vuls
go 1.20
require (
github.com/3th1nk/cidr v0.2.0
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible
github.com/BurntSushi/toml v1.3.2
github.com/CycloneDX/cyclonedx-go v0.7.1

3
go.sum
View File

@@ -188,6 +188,8 @@ cloud.google.com/go/webrisk v1.5.0/go.mod h1:iPG6fr52Tv7sGk0H6qUFzmL3HHZev1htXuW
cloud.google.com/go/workflows v1.6.0/go.mod h1:6t9F5h/unJz41YqfBmqSASJSXccBLtD1Vwf+KmJENM0=
cloud.google.com/go/workflows v1.7.0/go.mod h1:JhSrZuVZWuiDfKEFxU0/F1PQjmpnpcoISEXH2bcHC3M=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/3th1nk/cidr v0.2.0 h1:81jjEknszD8SHPLVTPPk+BZjNVqq1ND2YXLSChl6Lrs=
github.com/3th1nk/cidr v0.2.0/go.mod h1:XsSQnS4rEYyB2veDfnIGgViulFpIITPKtp3f0VxpiLw=
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible h1:fcYLmCpyNYRnvJbPerq7U0hS+6+I79yEDJBqVNcqUzU=
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 h1:UQHMgLO+TxOElx5B5HZ4hJQsoJ/PvUvKRhJHDQXO8P8=
@@ -769,6 +771,7 @@ github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=