Synchronization service

Synchronization service#

Vulnerability-Lookup can synchronize objects (comments, bundles, sightings, KEV entries) from remote Vulnerability-Lookup instances. This allows a local instance to pull published data from one or more remote instances via their public APIs.

Configuration#

Remote instances are managed from the admin interface at /admin/remote_instances/. For each remote instance you can configure:

  • Name: a label for the remote instance.

  • Address: the base URL of the remote instance (e.g. https://vulnerability.circl.lu).

  • GNA Identifier: optional GCVE GNA identifier.

  • Sync toggles: choose which object types to synchronize (comments, bundles, sightings, KEV entries).

  • Sync local objects only: when enabled (default), only objects originally created on the remote instance are synced — objects the remote itself synced from elsewhere are skipped.

  • Active: enable or disable the instance.

The last sync date is displayed when editing a remote instance and can be reset to force a full re-pull on the next sync cycle.

Launching the service with poetry#

From the directory used to install Vulnerability-Lookup:

Daemon mode (runs every hour by default):

poetry run flask --app website.app sync

One-shot mode (runs once and exits, ideal for cron):

poetry run flask --app website.app sync --interval 0

Custom interval (e.g. every 30 minutes):

poetry run flask --app website.app sync --interval 1800

Specifying the sync user (synced objects will be attributed to this user):

poetry run flask --app website.app sync --login myuser

With file logging:

poetry run flask --app website.app sync --log-file /var/log/vulnerability-lookup_sync.log

Launching the service with systemd#

Create the file /etc/systemd/system/vulnerability-lookup-sync.service:

[Unit]
Description=Vulnerability-Lookup sync service
After=vulnerability-lookup-web.service
Requires=vulnerability-lookup-web.service

[Service]
Type=simple
User=<system user used to install Vulnerability-Lookup>
Group=<group of the user used to install Vulnerability-Lookup>
WorkingDirectory=<path to the cloned repository>
Environment="VULNERABILITYLOOKUP_HOME=<path to the cloned repository>"
ExecStart=<path to poetry> run python -m flask \
    --app website.app sync \
    --login "sync_user" \
    ----interval 600 \
    --log-file /var/log/vulnerability-lookup/sync.log
StandardOutput=append:/var/log/vulnerability-lookup_sync_message.log
StandardError=append:/var/log/vulnerability-lookup_sync_error.log
Restart=on-failure
RestartSec=15s

[Install]
WantedBy=multi-user.target

Note

Adjust the variables appropriately. A sample service file is also available at etc/systemd/system/vulnerability-lookup-sync.service.sample.

Then reload and enable the service:

sudo systemctl daemon-reload
sudo systemctl enable vulnerability-lookup-sync.service
sudo systemctl start vulnerability-lookup-sync.service
systemctl status vulnerability-lookup-sync.service

To follow the service logs live:

sudo journalctl -u vulnerability-lookup-sync.service -f

Error logs:

tail -f /var/log/vulnerability-lookup_sync_error.log

Sync logs:

tail -f /var/log/vulnerability-lookup_sync.log

Batch deletion of synced objects#

Synced objects can be deleted in bulk via the API using the vulnerability_lookup_origin parameter, which corresponds to the UUID of the remote instance they were pulled from.

# Delete all comments from a specific remote instance
curl -X DELETE -H "X-API-Key: <api_key>" \
    "https://localhost:10001/api/comment/?vulnerability_lookup_origin=<remote-uuid>"

# Delete all bundles from a specific remote instance
curl -X DELETE -H "X-API-Key: <api_key>" \
    "https://localhost:10001/api/bundle/?vulnerability_lookup_origin=<remote-uuid>"

# Delete all sightings from a specific remote instance
curl -X DELETE -H "X-API-Key: <api_key>" \
    "https://localhost:10001/api/sighting/?vulnerability_lookup_origin=<remote-uuid>"
Contents