abdulhade/db-middleware

Fork 0

Files

abdulhade 7e7c22d0bb ADDED EMOJIS TO THE DOCS, WOW 😯

2025-04-07 14:40:19 +03:00

16 KiB

Raw Blame History

🏦 Capital Index Database Middleware

📄 Overview

💡 About This Project

The Capital Index Database Middleware provides an abstraction layer for your database, enhancing query management and change monitoring. It streamlines database interactions while ensuring efficient performance and security.

📁 Repository Contents

This repository includes:

Middleware Source Code – The core logic for database interaction.
Dockerfile – Configuration for containerized deployment.
Install & Management Scripts – Scripts for streamlined installation and maintenance.

Check Legal & Licensing

🧭 Table of Content:

🏦 Capital Index Database Middleware

🚀 Getting Started

🛠️ Installation

🔧 Step 1: Install Required Dependencies

Before installing, ensure you have the necessary dependencies to download and build the Docker container.

Debian/Ubuntu

sudo apt-get update && sudo apt-get upgrade -y
sudo apt-get install -y wget git docker.io

[Optional] Install Docker BuildKit:

sudo apt-get install -y docker-buildx

CentOS/Fedora

Note: If using CentOS 7, replace dnf with yum.

sudo dnf update -y && sudo dnf upgrade -y
sudo dnf install -y wget git docker

[Optional] Install Docker BuildKit:

sudo dnf install -y docker-buildx

Arch Linux

sudo pacman -Syu --noconfirm
sudo pacman -S --noconfirm wget git docker

[Optional] Install Docker BuildKit:

sudo pacman -S --noconfirm docker-buildx

Alpine Linux

sudo apk update && sudo apk upgrade
sudo apk add --no-cache wget git docker

[Optional] Install Docker BuildKit:

sudo apk add --no-cache docker-cli-buildx

▶️ Step 2: Run the Setup Script

After installing dependencies, configure Docker permissions and run the setup script:

Add your user to the Docker group and re-login:

sudo usermod -aG docker $USER && sudo su - $USER

Download and execute the setup.sh script:

mkdir -p ~/.db-middleware/scripts \
&& rm -rf ~/.db-middleware/scripts/* \
&& cd ~/.db-middleware/scripts \
&& wget -qO setup.sh https://gitea.abdulhade.com/abdulhade/db-middleware/raw/branch/main/scripts/setup.sh \
&& bash setup.sh

Give the container permission to edit its files:

sudo groupadd -g 2000 dbmiddleware
sudo usermod -aG dbmiddleware $USER
sudo chown -R $USER:dbmiddleware /home/$USER/.db-middleware/files
sudo chmod -R 775 /home/$USER/.db-middleware/files

⚙️ Configuration

During installation, you will be prompted to configure the following variables:

🔢 API_PORT

Defines the port the middleware listens on.
Default: 8080
Access within your machine: http://localhost:8080
Ensure this port is forwarded if external access is required.

📦 CONTAINER_NAME

Specifies the name of the Docker container running the middleware.

💾 HAS_LOCAL_DBS

Determines how the middleware connects to databases:

Enter 0 – If all the databases are hosted on a remote server.
Enter 1 – If one or more of the database runs on the same machine.

Note: By default, Docker containers operate on an isolated network. If your database is local, the middleware must be configured to run on the same network to ensure connectivity.

✅ Test your setup

📊 Using the `status` command:

Check the status of the container

$ db-middleware status

If the container is stopped, run db-middleware start command.

It shows something like this:

Config file loaded successfully.

+------------------------------+
| Checking container status... |
+------------------------------+


+--------------------------------------------+
| Database Middleware Status:                |
|                                            |
|                                            |
| [Container]                                |
|                                            |
|   Name: con-db-middleware                  |
|   Status: Up 11 minutes                    |
|                                            |
| [Performance]                              |
|                                            |
|   CPU Usage: 0.33%                         |
|   Memory Usage: 62.27MiB / 3.63GiB (1.68%) |
|   Block I/O: 0B / 0B                       |
|   Network I/O: 6.34kB / 4.09kB             |
|                                            |
| [Network]                                  |
|                                            |
|   Network Mode: bridge                     |
|   IP Address: 172.17.0.2                   |
| 172.17.0.2                                 |
|   Ports: 0.0.0.0:8080->8080/tcp            |
|                                            |
| [App]                                      |
|                                            |
|   Run Command: "bash /app/scripts/run.sh"  |
+--------------------------------------------+

🧪 Using the `test_setup` command:

If the container is created and running, use the test setup command:

$ db-middleware test_setup

Config file loaded successfully.

+-------------------------------------------+
| Container 'con-db-middleware' is Found.   |
| Container 'con-db-middleware' is running. |
| App returned Ok to ping request.          |
+-------------------------------------------+

👤 Create your first admin user

First make sure it is running by calling the db-middleware start command.

We have two types of users, admin and regular (called just user), the admin can add/update/delete database connections, and create/delete normal users.

We recommend creating one admin user, and use that to create normal users via the API.

To create your first admin user, simply use the create_user command:

$ db-middleware create_user
Config file loaded successfully.
Enter username: myadmin
Enter role (admin/user): admin

+-----------------------------------------------------------------------------+
| > User 'myadmin' with role 'admin' created successfully.                    |
| > API Key: 4887f28e378dfd99e622833ccbebc174a45d                             |
+-----------------------------------------------------------------------------+

You don't need passwords for this, just the username and role.

🌍 Expose the API

This app currently runs on your local network only, we need to expose it so the data consumer on the Data Analysis product can access it and consume the data it provides.

IMPORTANT: You must implement TLS encryption, we will never use a non-encrypted connection to transfer your data!

You can use a dedicated domain, or a sub-domain of yours.

You can use any reverse proxy you prefer, we will guide you through Nginx and Apache.

🌐 Using Nginx

Nginx is a great reverse-proxy that supports connection polling and real-time connections like Webhook and Server Site Events.

Install & Update Nginx:

sudo apt install nginx --upgrade
Create the config file:

/etc/nginx/sites-available/your_domain_com.conf


server {

    server_name "your_domain.com";   # Replace with your domain
    
    listen 80;
    listen [::]:80; 
    client_max_body_size 10G;

    proxy_connect_timeout 7d;
    proxy_send_timeout    7d;
    proxy_read_timeout    7d;
    send_timeout          7d;
    keepalive_timeout     7d;

    proxy_buffer_size 128k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;

    proxy_set_header Accept-Encoding "";

    location / {
        proxy_pass http://localhost:8080; # Replace with the port you set [default 8080].
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Enable the Config


sudo ln -s /etc/nginx/sites-available/your_domain_com.conf /etc/nginx/sites-enabled/
sudo nginx -t  # Test config
sudo systemctl restart nginx

Test the API (HTTP)

On your browser, head to: http://your_domain.com/ping

It should show "Ok"
Implement TLS Encryption

You can use any certificate provider, we'll show you how to use Certbot Setup (Let’s Encrypt TLS) to obtain a free certificate.

Run this after Nginx is live on port 80 (ensure DNS points to your server!):
```
sudo apt install -y certbot python3-certbot-nginx 
sudo certbot --nginx -d your_domain.com         # Replace with your domain
```
Certbot will auto-configure Nginx to use HTTPS and redirect HTTP → HTTPS.

Certificates auto-renew (add a cron job if not existing):
```
sudo certbot renew --dry-run
```
Test the API (HTTP/TLS)

On your browser, head to: https://your_domain.com/ping

It should show "Ok"

Notice the s in https

🅰️ Using Apache

Make sure apache is installed and upgraded

sudo apt install apache2 --upgrade -y

Create a new config file for your domain

sudo nano /etc/apache2/sites-available/your_domain.conf

Adjust your_domain.com and port 8080 if needed.

<VirtualHost *:80>
   # Adjust your domain
   ServerName your_domain.com

   LimitRequestBody 10737418240

   Timeout 604800
   ProxyTimeout 604800
   KeepAlive On
   KeepAliveTimeout 604800

   ProxyPreserveHost On

   # Adjust the port if needed
   ProxyPass / http://localhost:8080/
   ProxyPassReverse / http://localhost:8080/

   # Headers for WebSockets/real IP
   RequestHeader set Upgrade %{HTTP_UPGRADE}e
   RequestHeader set Connection "upgrade"
   RequestHeader set Host "%{Host}i"
   RequestHeader set X-Real-IP "%{REMOTE_ADDR}e"
   RequestHeader set X-Forwarded-For "%{REMOTE_ADDR}e"

   # Disable compression (if needed for SSE)
   RequestHeader unset Accept-Encoding

   # Buffer optimizations (similar to Nginx)
   ProxyIOBufferSize 131072
</VirtualHost>

Enable required Apache modules

sudo a2enmod proxy proxy_http headers
Enable your site

sudo a2ensite your_domain.conf

Test & Reload Apache

sudo apache2ctl configtest  # Should say "Syntax OK"
sudo systemctl reload apache2

Implement TLS Encryption

You can use any certificate provider, we'll show you how to use Certbot Setup (Let’s Encrypt TLS) to obtain a free certificate.
```
sudo apt install certbot python3-certbot-apache -y
sudo certbot --apache -d your_domain.com # Replace with your domain
```
Certbot will auto-configure Nginx to use HTTPS and redirect HTTP → HTTPS.

Certificates auto-renew (add a cron job if not existing):
```
sudo certbot renew --dry-run
```

🚀 Available Commands

Organized by functionality to help you navigate easily.

🛠️ Source Code Management

Command	Description
`install`	Initializes the environment using `setup.sh`. Builds the image and setup scripts.
`update_code`	Updates local code (`~/.db-middleware/code`) and inside the container.
`upgrade`	Pulls latest code, rebuilds image, replaces container if updates exist.
`rebuild`	Rebuilds the image with the current local repository code.

🚦 App Running

Command	Description
`status`	Displays container status and related info.
`test_setup`	Validates setup: checks image, container, and ping endpoint.
`start`	Starts the container if stopped or creates it if missing.
`restart`	Restarts the container or creates a new one if not found.
`stop`	Stops the container if it's currently running.

👥 User Management

Command	Description
`create_user`	Creates a user. Prompts for username and role (`admin`/`user`).

⚙️ Configuration

Command	Description
`show_config`	Displays the current configuration settings.
`update_config`	Walks you through updating configuration values. See Configuration.

📚 Help

Command	Description
`help`	Lists all available commands with info.

Legal & Licensing

⚠️ Important Legal Notice

This software is NOT open source. You are not permitted to use, copy, modify, distribute, sublicense, or sell this software unless explicitly authorized in writing by the original author under a signed agreement.

By accessing, downloading, or using this software, you agree to the following:

You may only use this software in the context defined in your individual or organizational agreement with the author.
Redistribution, resale, reverse-engineering, or use outside the agreed scope is strictly prohibited.
Violating these terms — including using the code without permission, reselling it, or distributing it to others — will result in legal action, including but not limited to:
- Filing of lawsuits for intellectual property infringement.
- Demand for damages, including financial compensation and legal fees.

If you are unsure about your rights or agreement terms, stop using this code immediately and contact the author.

For further assistance, refer to the project documentation or open an issue in the repository.

16 KiB Raw Blame History Unescape Escape

🏦 Capital Index Database Middleware

📄 Overview

💡 About This Project

📁 Repository Contents

🧭 Table of Content:

🚀 Getting Started

🛠️ Installation

🔧 Step 1: Install Required Dependencies

▶️ Step 2: Run the Setup Script

⚙️ Configuration

🔢 API_PORT

📦 CONTAINER_NAME

💾 HAS_LOCAL_DBS

✅ Test your setup

📊 Using the status command:

🧪 Using the test_setup command:

👤 Create your first admin user

🌍 Expose the API

🌐 Using Nginx

🅰️ Using Apache

🚀 Available Commands

🛠️ Source Code Management

🚦 App Running

👥 User Management

⚙️ Configuration

📚 Help

Legal & Licensing

⚠️ Important Legal Notice

16 KiB

Raw Blame History

📊 Using the `status` command:

🧪 Using the `test_setup` command: