# Capital Index Database Middleware ## Overview ### About This Project The **Capital Index Database Middleware** provides an abstraction layer for your database, enhancing query management and change monitoring. It streamlines database interactions while ensuring efficient performance and security. ### Repository Contents This repository includes: - **Middleware Source Code** – The core logic for database interaction. - **Dockerfile** – Configuration for containerized deployment. - **Install & Management Scripts** – Scripts for streamlined installation and maintenance. ### Table of Content: - [Capital Index Database Middleware](#capital-index-database-middleware) - [Overview](#overview) - [About This Project](#about-this-project) - [Repository Contents](#repository-contents) - [Table of Content:](#table-of-content) - [Getting Started](#getting-started) - [Installation](#installation) - [Step 1: Install Required Dependencies](#step-1-install-required-dependencies) - [Step 2: Run the Setup Script](#step-2-run-the-setup-script) - [Configuration](#configuration) - [1. API\_PORT](#1-api_port) - [2. CONTAINER\_NAME](#2-container_name) - [3. HAS\_LOCAL\_DBS](#3-has_local_dbs) - [Test your setup](#test-your-setup) - [Using the `status` command:](#using-the-status-command) - [Using the `test_setup` command:](#using-the-test_setup-command) - [Create your first admin user](#create-your-first-admin-user) - [Expose the API](#expose-the-api) - [Using Nginx](#using-nginx) - [Using Apache](#using-apache) - [Legal \& Licensing](#legal--licensing) - [⚠️ Important Legal Notice](#️-important-legal-notice) ## Getting Started ### Installation #### Step 1: Install Required Dependencies Before installing, ensure you have the necessary dependencies to download and build the Docker container.
Debian/Ubuntu ```bash sudo apt-get update && sudo apt-get upgrade -y sudo apt-get install -y wget git docker.io ``` [Optional] Install Docker BuildKit: ```bash sudo apt-get install -y docker-buildx ```
CentOS/Fedora > **Note:** If using CentOS 7, replace `dnf` with `yum`. ```bash sudo dnf update -y && sudo dnf upgrade -y sudo dnf install -y wget git docker ``` [Optional] Install Docker BuildKit: ```bash sudo dnf install -y docker-buildx ```
Arch Linux ```bash sudo pacman -Syu --noconfirm sudo pacman -S --noconfirm wget git docker ``` [Optional] Install Docker BuildKit: ```bash sudo pacman -S --noconfirm docker-buildx ```
Alpine Linux ```bash sudo apk update && sudo apk upgrade sudo apk add --no-cache wget git docker ``` [Optional] Install Docker BuildKit: ```bash sudo apk add --no-cache docker-cli-buildx ```
#### Step 2: Run the Setup Script After installing dependencies, configure Docker permissions and run the setup script: 1. Add your user to the Docker group and re-login: ```bash sudo usermod -aG docker $USER && sudo su - $USER ``` 2. Download and execute the `setup.sh` script: ```bash mkdir -p ~/.db-middleware/scripts \ && rm -rf ~/.db-middleware/scripts/* \ && cd ~/.db-middleware/scripts \ && wget -qO setup.sh https://gitea.abdulhade.com/abdulhade/db-middleware/raw/branch/main/scripts/setup.sh \ && bash setup.sh ``` 3. Give the container permission to edit its files: ```bash sudo groupadd -g 2000 dbmiddleware sudo usermod -aG dbmiddleware $USER sudo chown -R $USER:dbmiddleware /home/$USER/.db-middleware/files sudo chmod -R 775 /home/$USER/.db-middleware/files ``` --- ## Configuration During installation, you will be prompted to configure the following variables: ### 1. API_PORT - Defines the port the middleware listens on. - **Default:** `8080` - Access within your machine: `http://localhost:8080` - Ensure this port is forwarded if external access is required. ### 2. CONTAINER_NAME - Specifies the name of the Docker container running the middleware. ### 3. HAS_LOCAL_DBS Determines how the middleware connects to databases: - **Enter `0`** – If all the databases are hosted on a remote server. - **Enter `1`** – If one or more of the database runs on the same machine. > **Note:** By default, Docker containers operate on an isolated network. If your database is local, the middleware must be configured to run on the same network to ensure connectivity. ## Test your setup ### Using the `status` command: Check the status of the container ```bash $ db-middleware status ``` If the container is stopped, run `db-middleware start` command.
It shows something like this: ``` Config file loaded successfully. +------------------------------+ | Checking container status... | +------------------------------+ +--------------------------------------------+ | Database Middleware Status: | | | | | | [Container] | | | | Name: con-db-middleware | | Status: Up 11 minutes | | | | [Performance] | | | | CPU Usage: 0.33% | | Memory Usage: 62.27MiB / 3.63GiB (1.68%) | | Block I/O: 0B / 0B | | Network I/O: 6.34kB / 4.09kB | | | | [Network] | | | | Network Mode: bridge | | IP Address: 172.17.0.2 | | 172.17.0.2 | | Ports: 0.0.0.0:8080->8080/tcp | | | | [App] | | | | Run Command: "bash /app/scripts/run.sh" | +--------------------------------------------+ ```
### Using the `test_setup` command: If the container is created and running, use the test setup command: ```bash $ db-middleware test_setup ``` ```bash Config file loaded successfully. +-------------------------------------------+ | Container 'con-db-middleware' is Found. | | Container 'con-db-middleware' is running. | | App returned Ok to ping request. | +-------------------------------------------+ ``` ## Create your first admin user First make sure it is running by calling the `db-middleware start` command. We have two types of users, admin and regular (called just user), the admin can add/update/delete database connections, and create/delete normal users. We recommend creating one admin user, and use that to create normal users via the API. To create your first admin user, simply use the `create_user` command: ```bash $ db-middleware create_user Config file loaded successfully. Enter username: myadmin Enter role (admin/user): admin +-----------------------------------------------------------------------------+ | > User 'myadmin' with role 'admin' created successfully. | | > API Key: 4887f28e378dfd99e622833ccbebc174a45d | +-----------------------------------------------------------------------------+ ``` You don't need passwords for this, just the username and role. ## Expose the API This app currently runs on your local network only, we need to expose it so the data consumer on the Data Analysis product can access it and consume the data it provides. > **IMPORTANT:** You must implement TLS encryption, we will never use a non-encrypted connection to transfer your data! You can use a dedicated domain, or a sub-domain of yours. You can use any reverse proxy you prefer, we will guide you through Nginx and Apache. ### Using Nginx Nginx is a great reverse-proxy that supports connection polling and real-time connections like Webhook and Server Site Events. 1. Install & Update Nginx: `sudo apt install nginx --upgrade` 2. Create the config file: `/etc/nginx/sites-available/your_domain_com.conf` ```bash server { server_name "your_domain.com"; # Replace with your domain listen 80; listen [::]:80; client_max_body_size 10G; proxy_connect_timeout 7d; proxy_send_timeout 7d; proxy_read_timeout 7d; send_timeout 7d; keepalive_timeout 7d; proxy_buffer_size 128k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k; proxy_set_header Accept-Encoding ""; location / { proxy_pass http://localhost:8080; # Replace with the port you set [default 8080]. proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } } ``` 1. Enable the Config ```bash sudo ln -s /etc/nginx/sites-available/your_domain_com.conf /etc/nginx/sites-enabled/ sudo nginx -t # Test config sudo systemctl restart nginx ``` 2. Test the API (HTTP) On your browser, head to: `http://your_domain.com/ping` It should show **"Ok"** 3. Implement TLS Encryption You can use any certificate provider, we'll show you how to use Certbot Setup (Let’s Encrypt TLS) to obtain a free certificate. Run this after Nginx is live on port 80 (ensure DNS points to your server!): ```bash sudo apt install -y certbot python3-certbot-nginx sudo certbot --nginx -d your_domain.com # Replace with your domain ``` Certbot will auto-configure Nginx to use HTTPS and redirect HTTP → HTTPS. Certificates auto-renew (add a cron job if not existing): ```bash sudo certbot renew --dry-run ``` 4. Test the API (HTTP/TLS) On your browser, head to: `https://your_domain.com/ping` It should show **"Ok"** > Notice the **s** in http**s** ### Using Apache 1. Make sure apache is installed and upgraded `sudo apt install apache2 --upgrade -y ` 2. Create a new config file for your domain `sudo nano /etc/apache2/sites-available/your_domain.conf` > Adjust `your_domain.com` and port `8080` if needed. ``` # Adjust your domain ServerName your_domain.com LimitRequestBody 10737418240 Timeout 604800 ProxyTimeout 604800 KeepAlive On KeepAliveTimeout 604800 ProxyPreserveHost On # Adjust the port if needed ProxyPass / http://localhost:8080/ ProxyPassReverse / http://localhost:8080/ # Headers for WebSockets/real IP RequestHeader set Upgrade %{HTTP_UPGRADE}e RequestHeader set Connection "upgrade" RequestHeader set Host "%{Host}i" RequestHeader set X-Real-IP "%{REMOTE_ADDR}e" RequestHeader set X-Forwarded-For "%{REMOTE_ADDR}e" # Disable compression (if needed for SSE) RequestHeader unset Accept-Encoding # Buffer optimizations (similar to Nginx) ProxyIOBufferSize 131072 ``` 3. Enable required Apache modules `sudo a2enmod proxy proxy_http headers` 4. Enable your site `sudo a2ensite your_domain.conf` 5. Test & Reload Apache ``` sudo apache2ctl configtest # Should say "Syntax OK" sudo systemctl reload apache2 ``` 6. Implement TLS Encryption You can use any certificate provider, we'll show you how to use Certbot Setup (Let’s Encrypt TLS) to obtain a free certificate. ```bash sudo apt install certbot python3-certbot-apache -y sudo certbot --apache -d your_domain.com # Replace with your domain ``` Certbot will auto-configure Nginx to use HTTPS and redirect HTTP → HTTPS. Certificates auto-renew (add a cron job if not existing): ```bash sudo certbot renew --dry-run ``` ## Legal & Licensing ### ⚠️ Important Legal Notice This software is NOT open source. You are not permitted to use, copy, modify, distribute, sublicense, or sell this software unless explicitly authorized in writing by the original author under a signed agreement. By accessing, downloading, or using this software, you agree to the following: * You may only use this software in the context defined in your individual or organizational agreement with the author. * Redistribution, resale, reverse-engineering, or use outside the agreed scope is strictly prohibited. * Violating these terms — including using the code without permission, reselling it, or distributing it to others — will result in legal action, including but not limited to: * Filing of lawsuits for intellectual property infringement. * Demand for damages, including financial compensation and legal fees. **If you are unsure about your rights or agreement terms, stop using this code immediately and contact the author.** --- For further assistance, refer to the project documentation or open an issue in the repository.