Configuration

The protea.config subpackage provides tuning parameters and configuration helpers used across the PROTEA stack.

Tuning parameters

protea.config.tuning exposes TuningSettings, a Pydantic settings class that aggregates knobs for batch sizes, timeouts, and algorithm parameters. Values are read from environment variables (with PROTEA_ prefix) and fall back to documented defaults. Workers instantiate a single TuningSettings object at startup; operations receive it via dependency injection rather than importing it directly, keeping them independently testable.

Runtime tuning settings (T-CONF.2).

Externalises hardcoded module-level constants from protea/ so an operator can tune throughput, retry policy and timeouts per deployment target (dev, prod-cloud, hpc-bsc, hpc-airgap) without touching code.

Hierarchy (lowest to highest priority):

1. Defaults baked into the pydantic models below.

2. tuning: section in protea/config/system.yaml.

3. Environment variables of the form PROTEA_TUNING__<group>__<field>.

Currently scoped to the QueueTuning group as a proof of concept. The remaining categories from docs/CONFIG_INVENTORY.md (WorkerTuning, OperationTuning, APILimits, ResearchKnobs) follow the same pattern and will be added incrementally.

Example:

from protea.config.tuning import get_tuning

settings = get_tuning()
for attempt in range(settings.queue.publisher_max_attempts):
    ...

class protea.config.tuning.APILimits(*, max_fasta_bytes: Annotated[int, Ge(ge=1024)] = 52428800, max_comment_length: Annotated[int, Ge(ge=1)] = 500, recent_limit: Annotated[int, Ge(ge=1)] = 20, page_limit: Annotated[int, Ge(ge=1)] = 100)

Bases: BaseModel

HTTP boundary limits enforced at the FastAPI router layer.

Sources: api/routers/{annotate,query_sets,support}.py (ver docs/CONFIG_INVENTORY.md §D).

max_comment_length: int

max_fasta_bytes: int

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

page_limit: int

recent_limit: int

class protea.config.tuning.OperationTuning(*, annotation_chunk_size: Annotated[int, Ge(ge=100)] = 10000, stream_chunk_size: Annotated[int, Ge(ge=100)] = 2000, store_chunk_size: Annotated[int, Ge(ge=500)] = 10000, numpy_query_chunk: Annotated[int, Ge(ge=10)] = 500, ref_cache_freshness_seconds: Annotated[int, Ge(ge=0)] = 300, aspect_knn_workers: Annotated[int, Ge(ge=1)] = 3, gpu_busy_retry_seconds: Annotated[int, Ge(ge=1)] = 8)

Bases: BaseModel

Module-level chunk and batch sizes used inside operations.

HTTP retry policy and per-source timeouts live inside their respective pydantic payloads (InsertProteinsPayload, LoadGoaAnnotationsPayload, etc.) because the caller picks them per-job. The values here are infra-level: how to slice work between memory and broker pressure constraints.

Sources: core/feature_enricher.py, core/knn_search.py, core/operations/{predict_go_terms,training_dump_helpers}.py (ver docs/CONFIG_INVENTORY.md §C).

annotation_chunk_size: int

aspect_knn_workers: int

gpu_busy_retry_seconds: int

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

numpy_query_chunk: int

ref_cache_freshness_seconds: int

store_chunk_size: int

stream_chunk_size: int

class protea.config.tuning.QueueTuning(*, publisher_max_attempts: Annotated[int, Ge(ge=1)] = 12, publisher_base_delay: Annotated[float, Ge(ge=0.0)] = 1.0, oom_max_retries: Annotated[int, Ge(ge=0)] = 5, oom_base_delay: Annotated[int, Ge(ge=0)] = 5, oom_max_delay: Annotated[int, Ge(ge=1)] = 300, amqp_heartbeat: Annotated[int, Ge(ge=0)] = 600)

Bases: BaseModel

RabbitMQ publisher / consumer retry and dispatch knobs.

Sources: infrastructure/queue/publisher.py and infrastructure/queue/consumer.py (ver docs/CONFIG_INVENTORY.md §A).

amqp_heartbeat: int

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

oom_base_delay: int

oom_max_delay: int

oom_max_retries: int

publisher_base_delay: float

publisher_max_attempts: int

class protea.config.tuning.ServeTuning(*, default_embedding_config_id: str | None = None, compute_alignments: bool = True, compute_taxonomy: bool = True, compute_v6_features: bool = False, compute_lineage_features: bool = False, interpro_bp_graft: bool = False, interpro_bp_graft_weight: Annotated[float, Ge(ge=0.0), Le(le=1.0)] = 0.5, interpro_bp_graft_source_version: str | None = None, classifier_impl_by_category: bool = False)

Bases: BaseModel

Live serving-path knobs for the one-click /annotate endpoint.

These pin the validated offline pipeline onto the serving path without hardcoding ids or flipping behaviour. Every default reproduces the pre-existing serve behaviour exactly, so a deployment that sets none of these env vars serves identically to before. To activate the validated pipeline at deploy time, set:

• default_embedding_config_id to the learned k-WTA retrieval config,

• compute_v6_features and compute_lineage_features to True so the predict payload matches the validated reranker feature schema,

• interpro_bp_graft to True for the InterPro2GO BP enrichment.

Sources: api/routers/annotate.py and the predict post-KNN pipeline.

classifier_impl_by_category: bool

compute_alignments: bool

compute_lineage_features: bool

compute_taxonomy: bool

compute_v6_features: bool

default_embedding_config_id: str | None

interpro_bp_graft: bool

interpro_bp_graft_source_version: str | None

interpro_bp_graft_weight: float

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class protea.config.tuning.TuningSettings(*, queue: QueueTuning = <factory>, worker: WorkerTuning = <factory>, operation: OperationTuning = <factory>, api: APILimits = <factory>, serve: ServeTuning = <factory>)

Bases: BaseModel

Root tuning model that composes per-category sub-models.

api: APILimits

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

operation: OperationTuning

queue: QueueTuning

serve: ServeTuning

worker: WorkerTuning

class protea.config.tuning.WorkerTuning(*, db_pool_size: Annotated[int, Ge(ge=1)] = 20, db_pool_max_overflow: Annotated[int, Ge(ge=0)] = 40, db_pool_recycle_seconds: Annotated[int, Ge(ge=60)] = 3600, model_cache_max: Annotated[int, Ge(ge=1)] = 1, ref_cache_max: Annotated[int, Ge(ge=1)] = 1, reaper_main_timeout_seconds: Annotated[int, Ge(ge=300)] = 21600, reaper_default_timeout_seconds: Annotated[int, Ge(ge=300)] = 3600, reaper_stall_seconds: Annotated[int, Ge(ge=60)] = 1800, reaper_event_grace_seconds: Annotated[int, Ge(ge=60)] = 2700, worker_shutdown_grace_seconds: Annotated[int, Ge(ge=1)] = 30, job_heartbeat_interval_seconds: Annotated[int, Ge(ge=5)] = 30, max_lease_requeues: Annotated[int, Ge(ge=0)] = 3, api_cache_default_ttl_seconds: Annotated[float, Ge(ge=1.0)] = 300.0)

Bases: BaseModel

Pool sizes, in-process caches and reaper timeouts.

Sources: infrastructure/database/engine.py, infrastructure/operations/{compute_embeddings,predict_go_terms}.py, workers/stale_job_reaper.py, api/cache.py (ver docs/CONFIG_INVENTORY.md §B).

api_cache_default_ttl_seconds: float

db_pool_max_overflow: int

db_pool_recycle_seconds: int

db_pool_size: int

job_heartbeat_interval_seconds: int

max_lease_requeues: int

model_cache_max: int

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

reaper_default_timeout_seconds: int

reaper_event_grace_seconds: int

reaper_main_timeout_seconds: int

reaper_stall_seconds: int

ref_cache_max: int

worker_shutdown_grace_seconds: int

protea.config.tuning.get_tuning() → TuningSettings

Load and cache the tuning settings.

Cache reset (mostly for tests):

get_tuning.cache_clear()

See also

• Configuration Reference: full environment-variable reference.

• Infrastructure: protea.infrastructure.settings for the database and AMQP connection strings.