Configuration

Filomena uses .NET configuration. A setting named State:Provider becomes Filomena__State__Provider as an environment variable.

Core settings

Setting	Default	Required	Purpose
WebhookToken	none	yes	Compatibility bearer token used when a scoped token is absent
IngestToken	WebhookToken	no	Token for alert and change ingestion
ReadToken	WebhookToken	no	Token for read-only run, incident, enrichment and coverage APIs
AdminToken	WebhookToken	no	Token for feedback, verification and re-triage writes
AnalysisBaseUrl	https://api.openai.com/v1/	yes	OpenAI-compatible API base URL
AnalysisApiKey	empty	no	Bearer token for the analysis provider
AnalysisApiKeyHeader	empty	no	Custom API-key header; empty sends AnalysisApiKey as a bearer token
AnalysisPathTemplate	chat/completions	yes	Relative completion path; {model} is replaced with the escaped model name
AnalysisModelInBody	true	yes	Include model in the request body
AnalysisMaxTokensProperty	max_tokens	yes	Token-limit property; PX Foundry routes use max_completion_tokens
AnalysisOrganization	empty	no	Optional OpenAI organization header
Model	gpt-4.1-mini	yes	Provider model identifier
FallbackModel	empty	no	Model tried once when the primary fails with a retryable upstream condition (429, 408, 500, 502, 503, 504). Empty disables it. A 400 is never retried, because a different model would fail the same request
WorkerCount	2	yes	Concurrent triage workers
QueueCapacity	100	yes	Maximum in-memory queue depth
DedupWindowMinutes	30	yes	Suppress repeat completed triage runs
MaxDiagnosticCharacters	40000	yes	Final structured prompt budget
MaxProcessingAttempts	3	yes	Maximum persisted attempts per event
RetryBaseDelaySeconds	30	yes	Exponential retry base delay
RecoveryWindowHours	24	yes	Maximum age for retry recovery

Economics

Setting	Default	Purpose
Economics:InputUsdPerMillionTokens	0	Provider price for one million input tokens
FallbackEconomics:InputUsdPerMillionTokens	primary rate	Price of one million input tokens for FallbackModel
ModelEconomics:<model>:InputUsdPerMillionTokens	unset	Per-model input rate, read only by Filomena.Eval benchmark. A model with no rate reports no cost, and cost has no winner unless every compared model has rates
ModelEconomics:<model>:OutputUsdPerMillionTokens	unset	Per-model output rate for the same command
FallbackEconomics:OutputUsdPerMillionTokens	primary rate	Price of one million output tokens for FallbackModel. Unset in code costs fallback runs at the primary rates; Compose passes the primary values through explicitly, so set both when the fallback is dearer or its cost is under-reported
Economics:OutputUsdPerMillionTokens	0	Provider price for one million output tokens
Economics:FixedMonthlyUsd	0	Monthly infrastructure and fixed operating cost used in TCO reporting

The OpenAI-compatible provider response must include usage.prompt_tokens and usage.completion_tokens, or their input_tokens and output_tokens equivalents, for cost accounting. Missing usage stays zero rather than being estimated from characters.

Providers without bearer authentication can leave AnalysisApiKey empty. The endpoint must implement POST /chat/completions and accept response_format with a JSON schema.

SQLite backups

Setting	Default	Purpose
Backup:Enabled	false	Enable verified online SQLite backups
Backup:Directory	/var/backups/filomena	Absolute destination directory on separate durable storage
Backup:IntervalHours	24	Hours between backups
Backup:RetentionCount	14	Number of verified backups retained

Backups use SQLite's online backup API and run PRAGMA integrity_check before the temporary file is atomically renamed. The Helm chart requires backup.existingClaim when backups are enabled. Do not place that claim on the same storage backend and failure domain as the state claim.

Restore requires downtime because Filomena is a singleton. Set FILOMENA_BACKUP_CLAIM, set FILOMENA_RESTORE_IMAGE to an image reference pinned by digest, and run:

./deploy/restore-sqlite.sh filomena-20260822T120000000Z-<id>.db

The script scales the deployment to zero, restores the selected verified database, removes stale WAL sidecars, starts Filomena, and waits for readiness. Test restores regularly. A backup that has never been restored is unverified operationally.

State provider

State persistence is independent from telemetry storage.

Setting	Default	Supported values	Purpose
State:Provider	sqlite for a bare process; postgresql in Compose and Helm	sqlite, postgresql, clickhouse	State-store implementation
State:ConnectionString	provider default	provider connection string	Overrides the provider connection

PostgreSQL is the recommended production state provider:

Filomena__State__Provider=postgresql
Filomena__State__ConnectionString=Host=postgres;Port=5432;Database=filomena;Username=filomena;Password=secret

The PostgreSQL database must exist before Filomena starts. Filomena creates and versions its tables inside that database. Supply the connection string through a secret in production.

The SQLite default connection string is Data Source=data/filomena.db. Mount the database directory on durable storage.

The SQLite provider applies transactional, numbered schema migrations before accepting traffic and puts the database in write-ahead logging mode on first use, so a reader never blocks the writer. A database created by an earlier version is converted on startup. WAL keeps two sidecar files next to the database, filomena.db-wal and filomena.db-shm, so mount the directory rather than the single file. The online backup API captures a consistent database without copying WAL files directly. An in-memory database is left alone, because journalling does not apply to it.

For ClickHouse:

Filomena__State__Provider=clickhouse
Filomena__State__ConnectionString=Host=clickhouse;Port=8123;Username=filomena;Password=secret;Database=filomena;Protocol=http;Timeout=30

When State:ConnectionString is empty, the ClickHouse provider builds it from the legacy ClickHouseHost, ClickHousePort, ClickHouseUser and ClickHousePassword settings. ClickHouseDatabase selects the state database and defaults to filomena.

An unsupported provider fails startup.

Log evidence provider

Setting	Default	Supported values	Purpose
Logs:Provider	none	none, clickhouse, loki, datadog	Log and access-log evidence implementation. datadog supplies error logs only, not access-log summaries
Logs:ConnectionString	empty	provider connection string	ClickHouse OpenTelemetry database connection
Logs:BaseUrl	empty	absolute URL	Loki base URL, or the Datadog site API base such as https://api.datadoghq.com/
Logs:ApiKey	empty	string	Datadog API key, sent as DD-API-KEY. Required for the datadog provider
Logs:ApplicationKey	empty	string	Datadog application key, sent as DD-APPLICATION-KEY. Required for the datadog provider
Logs:Datadog:ServiceTag	service	string	Datadog tag carrying the service name
Logs:Datadog:NamespaceTag	kube_namespace	string	Datadog tag carrying the Kubernetes namespace
Logs:Datadog:PodTag	pod_name	string	Datadog tag carrying the pod name
Logs:Datadog:ContainerTag	kube_container_name	string	Datadog tag carrying the container name. container_name is Docker-only and is not applied to containerd-collected Kubernetes logs
Logs:TenantId	empty	string	Loki X-Scope-OrgID value
Logs:BearerToken	empty	secret	Loki bearer token
Logs:Username	empty	string	Loki basic-auth username
Logs:Password	empty	secret	Loki basic-auth password
Logs:ServiceLabel	service_name	label name	Loki service label
Logs:NamespaceLabel	namespace	label name	Loki Kubernetes namespace label
Logs:PodLabel	pod	label name	Loki Kubernetes pod label
Logs:ContainerLabel	container	label name	Loki Kubernetes container label
Logs:ApplicationGatewayService	azure-appgw	label or column value	Service value that carries Application Gateway logs, for Loki and ClickHouse

none disables log queries. Metrics, related alerts, prior cases and other diagnostic packs keep working.

For ClickHouse:

Filomena__Logs__Provider=clickhouse
Filomena__Logs__ConnectionString=Host=clickhouse;Port=8123;Username=filomena;Password=secret;Database=otel;Protocol=http;Timeout=30

When Logs:ConnectionString is empty, the ClickHouse provider uses the legacy ClickHouse settings with database otel. An unsupported provider fails startup.

For Loki:

Filomena__Logs__Provider=loki
Filomena__Logs__BaseUrl=https://loki.example.com
Filomena__Logs__TenantId=tenant-a
Filomena__Logs__BearerToken=secret

Loki queries use exact stream selectors from normalized alert labels, a bounded time range and a result limit. Basic auth takes precedence over bearer auth when Logs:Username is set. Application Gateway aggregation expects JSON log lines containing resource_name, listener_name, http_status, time_taken and server_response_latency either at the root or under LogAttributes, log_attributes or properties.

Slack

Slack is optional. Configure SlackBotToken and SlackChannelId to enable notifications. Configure SlackSigningSecret to enable interactive feedback:

Setting	Purpose
SlackBotToken	Bot token with chat:write
SlackChannelId	Destination channel ID
SlackPlatformChannelId	Compatibility fallback for the platform channel when SlackChannelId is empty
SlackLegacyChannelId	Optional channel for alerts carrying source=legacy
SlackCanaryChannelId	Optional isolated channel for alerts carrying filomena_canary=true
LegacyTriageEnabled	Whether alerts carrying source=legacy receive AI triage; defaults to true
SlackSigningSecret	Required for interactive feedback

Set the Slack app Interactivity Request URL to:

https://your-filomena-host/v1/slack/interactions

Filomena verifies X-Slack-Signature and X-Slack-Request-Timestamp. Configure ingress so only this path is public when the rest of the API is private.

ClickHouse log schema

Filomena expects otel.otel_logs with the OpenTelemetry fields declared in deploy/clickhouse/init/002-otel.sql.

The current query contract includes these materialized convenience columns:

• app
• level
• service_name
• http_status

When an existing OpenTelemetry schema does not provide them, add materialized columns or a compatible view.

Naming

Filomena matches some diagnostic packs on service and alert names. The defaults are the conventions Filomena was written against. Change them to your own without touching code.

Setting	Default	Purpose
Naming:EnvironmentSuffixes	dev, test, staging, prod, data, proddata	Trailing -<suffix> removed from a workload name before it becomes a pod or deployment selector
Naming:ApplicationGatewayServices	AppGateway	Alert service names that mean an Application Gateway
Naming:ApplicationGatewayAlertNames	AppGW	Alert-name fragments that mean an Application Gateway
Naming:MessageBrokerServices	ActiveMQ	Alert service names that mean an ActiveMQ or Artemis broker
Naming:JobAlertNames	Job failed	Alert names that mean a failed Kubernetes Job

A configured list replaces its default rather than adding to it, so a default can be turned off. An empty list disables that matching entirely, which environment variables cannot express: use a configuration file for that. An entry that is null or blank fails startup, because a list Filomena cannot read is operator error rather than a default. Service and alert-name matching is case-insensitive. A list uses the standard .NET array binding:

Filomena__Naming__EnvironmentSuffixes__0=qa
Filomena__Naming__EnvironmentSuffixes__1=sandbox
Filomena__Naming__ApplicationGatewayServices__0=edge-proxy

Label-based matching is not configurable, because label names come from the systems that export them: the ActiveMQ pack still matches a broker or queue label, the Application Gateway pack still matches a gateway or listener label, and the Job pack still matches a job_name label. Alert names that carry an upstream product name, such as ArgoCD and ExternalSecret, are also matched directly.

Metrics evidence provider

Setting	Default	Supported values	Purpose
Metrics:Provider	prometheus	none, prometheus	Metric evidence implementation
Metrics:BaseUrl	VictoriaMetricsUrl	absolute URL	Prometheus-compatible API base URL
Metrics:BearerToken	empty	secret	Optional bearer token

The provider uses /api/v1/query and /api/v1/query_range. VictoriaMetrics works through its Prometheus-compatible API. VictoriaMetricsUrl remains as a compatibility fallback when Metrics:BaseUrl is empty.

Diagnostic packs expect the metrics exported by the systems they cover. Missing metrics become collection errors and do not fail the entire run.

Kubernetes evidence provider

Setting	Default	Supported values	Purpose
Kubernetes:Provider	none	none, api	Kubernetes API evidence implementation
Kubernetes:BaseUrl	https://kubernetes.default.svc	absolute URL	Kubernetes API base URL
Kubernetes:BearerToken	empty	secret	Explicit bearer token
Kubernetes:TokenFile	service-account token path	file path	Rotating in-cluster token
Kubernetes:CertificateAuthorityFile	service-account CA path	file path	Kubernetes API CA certificate
Kubernetes:AllowInsecureTls	false	boolean	Disable TLS certificate validation

The API provider gets the alert pod, lists its events, requests previous logs for restarted containers and requests current logs for terminated containers. The service account needs this namespace-scoped RBAC:

rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["list"]

Argo CD evidence provider

Setting	Default	Supported values	Purpose
ArgoCd:Provider	none	none, api	Argo CD API evidence implementation
ArgoCd:BaseUrl	empty	absolute URL	Argo CD API base URL
ArgoCd:BearerToken	empty	secret	Argo CD API token

The provider gets /api/v1/applications/{name} and returns sync state, health, conditions, drifted resources and the latest operation. Give the token read-only applications, get permission for the projects and applications Filomena can triage.

Trace evidence provider

Setting	Default	Supported values	Purpose
Traces:Provider	none	none, tempo	Trace evidence implementation
Traces:BaseUrl	empty	absolute URL	Tempo HTTP API base URL
Traces:BearerToken	empty	secret	Optional Tempo bearer token

Tempo evidence searches traces using resource.service.name around the alert start. Service topology evidence reads the OpenTelemetry Collector service-graph metric traces_service_graph_request_total through the configured Prometheus provider.

Service identity

Filomena resolves one canonical identity from alert labels and annotations. Prefer these OpenTelemetry resource attributes:

• service.namespace
• service.name
• deployment.environment.name
• k8s.cluster.name
• k8s.namespace.name
• k8s.deployment.name

Optional operational metadata uses service.owner, service.repository and service.runbook. Existing service_name, environment, cluster, namespace, deployment, owner, repository and runbook_url labels remain supported.

Use incident.id, incident_id, incident_key or group_key to force alerts from multiple services into one incident. Without an explicit key, the canonical service namespace, name, environment and cluster form the correlation key.