Trading Sentiment Aggregator - Project Documentation

This repository contains an advanced financial data aggregator designed to scrape, compile, and structure market sentiment and fundamental analytics from diverse online sources. The application exposes a localized API framework built on FastAPI, incorporates a persistent SQLite database, and is containerized for seamless deployment via a dedicated CI/CD pipeline.

1. Software Architecture

The project is built around an automated multi-source ingestion pattern:

• Data Scrapers: Custom modular scraping routines designed to parse live history, analysis, and fundamental metrics from financial portals.
• Database Layer: A local SQLite database (aggregator.db) orchestrated through an asynchronous SQLAlchemy object-relational mapping architecture.
• API Framework: FastAPI endpoints providing organized read/write downstream access to consolidated ticker maps and aggregated market indexes.
• Containerization: Docker and Docker Compose definitions ensuring exact environment parity across development, staging, and final staging execution.

2. Directory Structure

trading-sentiment-aggregator/
├── api/
│   ├── deps.py
│   ├── endpoints.py
│   ├── __init__.py
│   └── main.py
├── core/
│   ├── config.py
│   ├── __init__.py
│   ├── security.py
│   └── ticker_map.py
├── data/
│   └── aggregator.db
├── database/
│   ├── base_model.py
│   ├── __init__.py
│   └── session.py
├── models
│   ├── __init__.py
│   ├── history.py
│   ├── instrument_cache.py
│   └── user.py
├── schemas
│   ├── __init__.py
│   ├── payload.py
│   ├── request.py
│   └── response.py
├── scripts
│   ├── manage_users.py
├── services
│   ├── __init__.py
│   ├── aggregator.py
│   ├── analyzer_client.py
│   ├── history_service.py
│   ├── news_provider.py
│   └── sentiment_service.py
├── tests
│   ├── __init__.py
│   ├── conftest.py
│   └── test_api.py
├── docker-compose.yml
├── Dockerfile
├── pytest.ini
├── README.md
└── requirements.txt

3. Technology Stack & Key Libraries

• Language: Python 3.11+
• Web Framework: FastAPI (Uvicorn server)
• Database & ORM: SQLite + SQLAlchemy (AsyncEngine implementation)
• Data Protocols: Protocol Buffers (pricing.proto) for structured payload operations
• Scraping Framework: Asynchronous requests and structured parsing libraries

4. API Endpoints

The API is served under the local application server root path and provides programmatic access to the ingestion network.

Core Endpoints:

• GET /docs: Interactive Swagger UI documentation
• POST /analyze: Analyze instrument

5. Usage

Send a request via POST to the /analyze endpoint: *

curl -X POST https://mcprod.icu/trading-sentiment-aggregator/analyze \
     -H "Content-Type: application/json" \
     -H "X-API-KEY: your-api-key" \
     -d '{"instrument": "TSLA", "lookback": 3}'

*to get an api-key send a mail to MC Sport+

You can expect a response such as

{
  "instrument": "USD",
  "current": {
    "headline": "RBC Capital Markets Says USD/CAD Bias Remains Skewed Toward Top End Of 1.3500-1.3900 Range",
    "prediction": "bearish",
    "confidence": 0.904,
    "source": "live"
  },
  "remainingCredits": 999999
}