# Configure NetoFuse

## Using the default configuration <a href="#using-the-default-configuration" id="using-the-default-configuration"></a>

The [Getting Started > Launch](/netofuse/get-started/launch.md) section provides the basic configuration steps to run a NetoFuse module.

## Where configuration is set <a href="#where-configuration-is-set" id="where-configuration-is-set"></a>

NetoFuse reads configurations in the following order:

1. `netofuse.yml` configuration file (use `--config` option to set a different filename/path)
2. `.netofuse.env` environment settings (use `--env` option to set a different filename/path)
3. Local environment settings

Environment variables override environment settings in a fille. Environment settings in a file override the YAML configuration file settings.

{% hint style="info" %}
**⚖️Choosing between YAML configuration and environment variables**

If you want to follow a simple best practice, use environment variables for credentials and the YAML configuration for all other settings.
{% endhint %}

## Environment settings <a href="#environment-settings" id="environment-settings"></a>

All configuration settings can be specified as environment variables that can be read from the local environment or an environment settings file.

### Start with the correct environment variable names <a href="#start-with-the-correct-environment-variable-names" id="start-with-the-correct-environment-variable-names"></a>

If you used the Docker host setup script, the correct environment variable names and default values for every setting are in the file: `/etc/netofuse/netofuse.env.template`

You can create this file at any time by running:

`netofuse init --envfile /app/netofuse/config/netofuse.env.template`

Remember, if you are using the container, file paths for netofuse command line arguments and settings refer to the location inside the container. Using the default setup script, `/app/netofuse/config` inside the container binds to `/etc/netofuse` on the Docker host.

#### Copying environment variables for all secrets to an environment file <a href="#copying-environment-variables-for-all-secrets-to-an-environment-file" id="copying-environment-variables-for-all-secrets-to-an-environment-file"></a>

This command will append all the sub-keys of all `credentials` keys to the `.netofuse.env` file.

`grep __CREDENTIALS__ /etc/netofuse/netofuse.env.template >> /etc/netofuse/.netofuse.env`

{% tabs %}
{% tab title="Environment" %}

```dotenv
NETO__API__APP_NAME=value   # this is the correct naming convention
NETO__API__APP__NAME=value  # this is incorrect, name is a subkey of app here
NETO_API_APP_NAME=value     # this is also incorrect, the whole string is 1 key
```

{% endtab %}

{% tab title="YAML" %}

{% endtab %}

{% tab title="Text" %}

{% endtab %}
{% endtabs %}

## Modifying `netofuse.yml` and specifying its location <a href="#modifying-netofuseyml-and-specifying-its-location" id="modifying-netofuseyml-and-specifying-its-location"></a>

NetoFuse looks in its current directory to find `netofuse.yml` unless its location is specified with one of:

* `--dir` command line option
* `--config` or `-c` command line option
* `NETOFUSE_CONF_PATH` environment variable

This is a YAML format configuration file.

{% hint style="info" %}
**📘What is YAML?**

"YAML is a human-readable data serialization language that is often used for writing configuration files. Depending on whom you ask, YAML stands for yet another markup language or YAML ain’t markup language (a recursive acronym), which emphasizes that YAML is for data, not documents."

Source: <https://www.redhat.com/en/topics/automation/what-is-yaml>
{% endhint %}

### If you are running a container <a href="#if-you-are-running-a-container" id="if-you-are-running-a-container"></a>

#### Using the provided docker run scripts <a href="#using-the-provided-docker-run-scripts" id="using-the-provided-docker-run-scripts"></a>

Default location for configuration files are `/etc/netofuse/netofuse.yml`and `/etc/netofuse/.netofuse.env`

By default, `.netofuse.env` does not exist - rather a template file `netofuse.env.template` is written to the directory to select the settings you want to use.

The run script will also pass all environment variables that start with the prefix `NETO__` on the Docker host to the container.

The docker run script provided in the [Install](/netofuse/get-started.md) page places a default copy of the configuration in `/etc/netofuse/` unless you modified the environment variables in that script before running it.

#### Advanced container configuration <a href="#advanced-container-configuration" id="advanced-container-configuration"></a>

If you are using your own `docker run` command and not using the docker run script provided, netofuse will look in its local directory inside the container for the default configuration file. Just placing the file in the local directory on the Docker host is not accessible within the container.

If you saved a copy of `netofuse.yml` on your docker host to `$PWD/config/netofuse.yml`, then you can run netofuse with this configuration by mapping that directory with docker as follows:

`docker run -v $PWD/config/:/app/netofuse/config public.ecr.aws/netography/netofuse:latest --config /app/netofuse/config/netofuse.yml`

If you’re using a different container orchestrator system like Kubernetes, supply a configmap of your configuration file, mount it inside the container, and supply arguments as you normally would.

### If you installed the Python package <a href="#if-you-installed-the--python-package" id="if-you-installed-the--python-package"></a>

#### Save a default configuration file (`netofuse init`) <a href="#save-a-default-configuration-file-netofuse-init" id="save-a-default-configuration-file-netofuse-init"></a>

Run `netofuse init` to save a default configuration file to `netofuse.yml` in your current directory.

#### Run netofuse with a specified configuration file <a href="#run-netofuse-with-a-specified-configuration-file" id="run-netofuse-with-a-specified-configuration-file"></a>

The `netofuse.yml` in the current directory when `netofuse` is run will be used as the configuration. You can use the `-c` or `--config` command line option to specify an alternate file path.

For example: `netofuse --config /etc/netofuse/netofuse.yml run axonius`

## `netofuse.yml` <a href="#netofuseyml" id="netofuseyml"></a>

The default `netofuse.yml` contains common settings and a default instance of each NetoFuse module. Each section of the file is documented in more detail below.

## `neto:` <a href="#neto" id="neto"></a>

The top-level section of a netofuse YAML file is `neto:`

{% tabs %}
{% tab title="YAML" %}

```
neto:
```

{% endtab %}
{% endtabs %}

### `syslog:` <a href="#syslog" id="syslog"></a>

This section defines the level of logging output and where that output is sent. The level field can be overridden with the `-l LEVEL` or `--level LEVEL` command line options (eg `netofuse -l DEBUG`).

{% tabs %}
{% tab title="YAML" %}

```
neto:
  syslog:                 # Default is log at INFO level to /dev/log
    level: VERBOSE        # Sets logging level (INFO,VERBOSE,WARNING,ERROR,DEBUG,TRACE)
    server: localhost     # Sends syslog to specified IP address if set (on port 514)
    address: /dev/log     # Sends syslog to specified LOG_DAEMON address if set (server has precedence)
    console: VERBOSE      # Sets a different logging level for console from syslog if set
```

{% endtab %}
{% endtabs %}

### `common:` <a href="#common" id="common"></a>

This section is applied to NetoFuse and the modules that execute:

{% tabs %}
{% tab title="YAML" %}

```
neto:
  common:
    retries: 3                      # Number of retries when a request fails
    retry_interval: 10    # Seconds to wait after first retry (exponential backoff is used)
    per_page: 9900        # Context labels to write per Netography API request
    upload: true          # Upload results to Netography
    cert: true            # Verify SSL certificates or path to cert
    container: true           # Running within a container
```

{% endtab %}
{% endtabs %}

### `save:` <a href="#save" id="save"></a>

Output from each module can be saved to files. This is useful for testing a module to see the output before uploading the results to the Netography Fusion API. It is also useful if you need to do any more advanced post-processing of data from a module before uploading it as an intermediary step, as you can save the output, edit it, and then use the `netofuse upload` command to upload the results.

Files will be saved to `modulename-output.ext`, `modulename-transform.ext`, and `modulename-labels.ext` where ext is the setting value (csv or json). If a path or filename is specified instead of just the extension, that wil be used instead of the default filenames.

{% tabs %}
{% tab title="YAML" %}

```
neto:
  save:
    output: csv     # if set to csv or json, saves pre-transform output from modules to local file
    transform: csv  # if set to csv or json, saves post-transform output from modules to local file
    labels: csv     # if set to csv or json, saves Netography context label format output from modules to local file
```

{% endtab %}
{% endtabs %}

### `cache:` <a href="#cache" id="cache"></a>

NetoFuse has 2 types of caching that can be enabled:

#### Context label caching <a href="#context-label-caching" id="context-label-caching"></a>

Caching context labels will create a file that stores a hash of the asset information that is being uploaded to Netography, and then only send changes in subsequent executions of the same module. It is recommended this is **NOT** enabled unless you have discussed the specific use case with Netography Support, as it is still in development.

#### Authentication token caching <a href="#authentication-token-caching" id="authentication-token-caching"></a>

If you make frequent successive calls to the Netography Fusion API, the authentication tokens used should be cached to improve performance and reduce load. With the token caching enabled, authentication can happen one time until the token expires, no matter how many different times NetoFuse is run in that period. If it is disabled, however, each time NetoFuse is run, it will request a new JWT request token, potentially creating many concurrent API sessions.

Adding a `cache:` section within a module configuration will override the global values.

{% tabs %}
{% tab title="YAML" %}

```
neto:
  cache:
    enabled: false  # caching of context labels for a module
    token: true     # caching of netography JWT tokens
    dir: cache/     # directory to store cached files in
```

{% endtab %}
{% endtabs %}

### `api:` <a href="#api" id="api"></a>

Defines the Netography API configuration parameters. If you are configuring netofuse for a single Netography API account, you can directly specify the values under the API key. If you are configuring multiple API instances (when you have multiple accounts within Netography Fusion and want to use a single configuration to manage and integrate them), give each instance a name and add values under that key.

Note: If you have multiple sub-accounts in Netography Fusion, you can generate an API key in the parent account and then change the `shortname` key to any sub-account that you want to use.

Example for a single account:

{% tabs %}
{% tab title="YAML" %}

```
neto:
  api:
    credentials:
      netosecret:                                       # Generated when creating API key
```

{% endtab %}
{% endtabs %}

Example when using multiple Netography API accounts:

{% tabs %}
{% tab title="YAML" %}

```
neto:
  api:
    api1:
      shortname:                                                    # Account name to use with API
      credentials:
        netosecret:                                         # Generated when creating API key
        api2:
      shortname:                                                    # Account name to use with API
      credentials:
        netosecret:                                         # Generated when creating API key
```

{% endtab %}
{% endtabs %}

Note: Instead of setting the `netosecret` value, you can instead set `url`, `app_name`, `shortname`,`credentials.app_key`, and `credentials.shared_secret`fields individually. This may be useful if you are using a legacy-style API key that predates the `netosecret` encoding format.

### Module Configuration <a href="#module-configuration" id="module-configuration"></a>

Each NetoFuse module is configured by the module name under the `neto` key .

The unique configuration parameters associated with each NetoFuse module are defined in its section of the [NetoFuse Modules](/netofuse/modules.md) documentation.

{% tabs %}
{% tab title="YAML" %}

```
neto:
  localfile:
    files: import/*   # Configuration parameter specific to the localfile module
```

{% endtab %}
{% endtabs %}

#### `transform:` <a href="#transform" id="transform"></a>

The transforms section for a module defines how keys from a module are mapped to context names in Fusion and how values are modified when they are written to context values in Fusion. Details on writing and editing transforms are documented in the [Context Transforms](/netofuse/context-transforms.md) section of the documentation.

Each module instance has a `transform` key set to a value of one of the transforms in this section (although individual modules may bypass this by setting `skip_transform: true` in the module configuration).

{% tabs %}
{% tab title="YAML" %}

```
neto:
  localfile:
    transform:
      ip:
        context: ip
      id:
        context: my_id
      name:
        context: name
      mac:
        context: mac_addr
```

{% endtab %}
{% endtabs %}

## Advanced Configuration Details <a href="#advanced-configuration-details" id="advanced-configuration-details"></a>

### Removing a configuration setting with UNSET <a href="#removing-a-configuration-setting-with-unset" id="removing-a-configuration-setting-with-unset"></a>

An environment variable that is set to the value `UNSET` will remove this setting entirely from the active configuration. This can be useful if you want to override a default configuration set in the YAML by removing it rather than changing the value.

### Complex configurations in environment variables <a href="#complex-configurations-in-environment-variables" id="complex-configurations-in-environment-variables"></a>

You can directly set any `transform` key in an environment variable to `YAML`. This allows you to store a complete transform, consisting of many sub-keys and values in a single environment variable.

You can also set any environment variables to other types (TOML, JSON) and have that automatically parsed when the variable is read at runtime.

### DynaConf library used for configuration <a href="#dynaconf-library-used-for-configuration" id="dynaconf-library-used-for-configuration"></a>

NetoFuse uses the DynaConf library for reading configuration. You can find detailed documentation on all the options for setting complex configuration values and how they are interpreted at:

<https://www.dynaconf.com/envvars/?h=environment>

### Mapping configuration settings to environment variable names manually <a href="#mapping-configuration-settings-to-environment-variable-names-manually" id="mapping-configuration-settings-to-environment-variable-names-manually"></a>

The environment variable name is the key to the configuration setting mapped to the YAML configuration file. Each level of YAML is delineated in the environment name by using 2 underscores `__` in the name.

Some configuration settings have a single underscore in the setting name. A single underscore is part of the key name. A double underscore splits the string that follows into a subkey of what precedes it.

For example, the setting `neto.api.url`maps to:

{% tabs %}
{% tab title="Environment" %}

```dotenv
NETO__API__URL=value
```

{% endtab %}

{% tab title="YAML" %}

{% endtab %}
{% endtabs %}

{% hint style="danger" %}
**❗️Environment variables can mix single and double underscores**

Some configuration settings may have a single underscore `_` in the setting name. A single underscore is part of the name of the configuration key, whereas a double underscore indicates what folllows is a sub-key (child, or new level) of the preceding key.

It can be easy when writing these names by hand to use `_` instead of `__` or vice-versa. For example, this is how 3 environment variable names with different use of underscores would map to different values in the YAML configuratio

{% tabs %}
{% tab title="Environment" %}

```dotenv
NETO__API__APP_NAME=value. # this is the correct naming convention
NETO__API__APP__NAME=value # this is incorrect, name is a subkey of app here
NETO_API_APP_NAME=value    # this is also incorrect, the whole string is 1 key
```

{% endtab %}

{% tab title="YAML" %}

{% endtab %}

{% tab title="Key Names" %}

{% endtab %}
{% endtabs %}
{% endhint %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.fusion.vectra.ai/netofuse/configure.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.