Update dev to main

.github/workflows/run-tests.yml

@@ -0,0 +1,33 @@
+name: Run Tests
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  run-tests:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v3
+
+    - name: Set up Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.x'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install -r test/requirements.txt
+
+    - name: Make test.sh executable
+      run: chmod +x test.sh
+
+    - name: Run test.sh
+      run: ./test.sh

.gitignore

@@ -1,3 +1,6 @@
 dist
 .pypirc
-struct
+struct
+*.log
+versions
+.mypy_cache

README.md

@@ -1,6 +1,6 @@
 # kvProcessor
 
-[**PYPI Package**](https://pypi.org/project/kvprocessor/) \
+[**PYPI Package**](https://pypi.org/project/kvprocessor/) **•** 
 [**GitHub**](https://github.com/connor33341/kvProcessor) \
 A Python package for processing and validating configuration dictionaries against a custom `.kv` file format.
 
@@ -14,86 +14,212 @@ pip install kvprocessor
 
 ## File format
 
+The `.kv` file format is a simple key-value configuration format with support for type validation and default values. Each line in a `.kv` file follows this syntax:
+
 ```custom
-# Comments are defined by a "#"
-VARIBLENAME<TYPE>:DEFAULTVAULE
+VARIABLENAME<TYPE>:DEFAULTVALUE
 ```
 
-## Usage
+- **VARIABLENAME**: The name of the variable.
+- **TYPE**: The expected type(s) of the variable. Multiple types can be separated by `|`.
+- **DEFAULTVALUE**: The default value for the variable. Use `none` if no default value is provided.
+- **Comments**: Both comments as a new line, or inline are supprorted, with the `#` character.
+
+### Example `.kv` file:
+```custom
+DATABASE_NAME<string>:none
+DATABASE_PORT<int>:3306
+ENABLE_LOGGING<bool>:true
+MAX_CONNECTIONS<int|none>:none
+```
+
+## KV Manifests
+
+A KV manifest is a file that defines namespaces and their relationships. It is used to organize and manage configurations across multiple `.kv` files. Each line in a manifest file follows this syntax:
+
+```custom
+namespace1:namespace2
+```
+
+- **namespace1**: The namespace that dosent exist as a file, but has the value as **namespace2**.
+- **namespace2**: The full namespace path
+
+### Example manifest file:
+```custom
+# A valid manifest
+root:database
+root:logging
+database:connection
+```
+
+### Validating a manifest:
+You can validate a manifest using the `KVManifestLoader`:
+
+```python
+from kvprocessor.kvmanifestloader import KVManifestLoader
+
+manifest_path = "test/manifest.txt"
+loader = KVManifestLoader(manifest_path)
+loader.validate_manifest()  # Validates the manifest structure
+```
+
+## Config.json
+
+The `config.json` file is used by the `KVStructLoader` to define the structure and metadata of the configuration. It includes details such as the version, root namespace, and manifest file.
 
+### Example `config.json`:
+**Note**: This example uses features from `0.1.10`, the current version is `0.2.14+`. Some extra parameters may be needed.
+```json
+{
+    "version": "0.1.10",
+    "root": "root",
+    "manifest": "manifest.txt",
+    "platform": "github",
+    "owner": "Voxa-Communications",
+    "repo": "VoxaCommunicaitons-Structures",
+    "branch": "main"
+}
+```
+
+### Using `KVStructLoader` with `config.json`:
 ```python
-from kvprocessor import LoadEnv, KVProcessor
+from kvprocessor import KVStructLoader
+
+kv_config_url = "https://github.com/Voxa-Communications/VoxaCommunicaitons-Structures/raw/refs/heads/main/struct/config.json"
+kv_struct_loader = KVStructLoader(kv_config_url)
+kv_processor = kv_struct_loader.from_namespace("root.database.connection")
+```
+
+## Usage
 
-kv_file_path = "test/test.kv" # Directory to .kv file
-kv_processor = KVProcessor(kv_file_path) # Create a KV processor class
-kv_keys = kv_processor.return_names() # Gets the keys (VARIBLENAME) from the .kv file
-env_list = LoadEnv(kv_keys) # Loads all the ENV varibles that match those keys
-validated_config = kv_processor.process_config(env_list) # Verifies that those env varibles exist and are of the correct type
+### KVProcessor
+```python
+from kvprocessor import KVProcessor
+from kvprocessor.kvenvloader import load_env
+
+kv_file_path = "test/test.kv"  # Directory to .kv file
+kv_processor = KVProcessor(kv_file_path)  # Create a KV processor class
+kv_keys = kv_processor.return_names()  # Gets the keys (VARIBLENAME) from the .kv file
+env_list = load_env(kv_keys)  # Loads all the ENV variables that match those keys
+validated_config = kv_processor.process_config(env_list)  # Verifies that those env variables exist and are of the correct type
 print(validated_config)
 ```
 
-This example mimics the one found in the `/test` directory. With the kv file of:
-```custom
-# This is a comment
-DATABASE_NAME<string>:none
-DATABASE_USER<string>:none
-DATABASE_PASSWORD<string>:none
-DATABASE_HOST<string>:none
-DATABASE_PORT<string|int>:none
-DATABASE_DRIVER<string>:mysql+mysqlconnector
-DATABASE_DIALECT<string>:none
-```
-You **should** get a result of: 
-`{'DATABASE_NAME': None, 'DATABASE_USER': None, 'DATABASE_PASSWORD': None, 'DATABASE_HOST': None, 'DATABASE_PORT': None, 'DATABASE_DRIVER': None, 'DATABASE_DIALECT': None}` This is because the kvProcessor is taking input from the env, and we dont have these env varibles defined. As a result these values default to the defined default value
-
-### Using "Namespaces"
-This allows you to "import" kv files from a static host.
+### KVStructLoader
 ```python
-from kvprocessor import KVProcessor, KVStructLoader
+from kvprocessor import KVStructLoader
 
-kv_config_url = "https://github.com/Voxa-Communications/VoxaCommunicaitons-Structures/raw/refs/heads/main/struct/config.json" # STATIC url to json config
-kv_struct_loader = KVStructLoader(kv_config_url) # Create a KVStructLoader object with the URL of the config file
-kv_processor: KVProcessor = kv_struct_loader.from_namespace("voxa.api.user.user_settings") # Loads the KV file from the URL and returns a KVProcessor object
-# For example this loads a file in /api/user/user_settings.kv
+kv_config_url = "https://github.com/Voxa-Communications/VoxaCommunicaitons-Structures/raw/refs/heads/main/struct/config.json"
+kv_struct_loader = KVStructLoader(kv_config_url)
+kv_processor = kv_struct_loader.from_namespace("root.database.connection")
 user_settings = {
-    "2FA_ENABLED": True,
-    "TELEMETRY": False,
-    "AGE": "25",
-    "LANGUAGE": "en",
-} # Example Dict Structure
+    "DATABASE_NAME": "test_db",
+    "DATABASE_PORT": 5432,
+}
 validated_config = kv_processor.process_config(user_settings)
 print(validated_config)
 ```
-For an example config.json navigate to `test/config.json`. This file is just what is found on `https://github.com/Voxa-Communications/VoxaCommunicaitons-Structures/blob/main/struct/config.json` which is used in this example.
 
-### Namespace's config.json
-Namespace JSON files, have to be on a static host. They cannot be used locally. The easiest way to do this is to make a github repo, and use the raw file.
-#### A config.json in a namespace should include at least two parts:
- - A "root", the name that preceedes the rest of the namespace. Ex: `voxa` in `voxa.api.user.user_settings`
- - A "URL". Ex: `https://mysite.example/kvstructures`, when the namespace `mysite.folder.structure` is used (assuming `root` is set to `mysite`), will fetch `https://mysite.example/kvstructures/folder/structure.kv`
+### KVFileMerger
+```python
+from kvprocessor import KVFileMerger
 
- Here is an example JSON (Note: on 0.7.1+ the URL is not needed, however a `struct` needs to be defined):
-```json
-{
-    "root": "voxa",
-    "version": "0.1.5",
-    "URL": "https://raw.githubusercontent.com/Voxa-Communications/VoxaCommunicaitons-Structures/refs/heads/main/struct/"
+file1 = "test/file1.kv"
+file2 = "test/file2.kv"
+merger = KVFileMerger(file1, file2)
+merged_file = merger.merge("merged.kv")  # Merges two KV files into a new file
+print(f"Merged file created at: {merged_file}")
+```
+
+### KVFileUtils
+```python
+from kvprocessor.kvfileutils import search_kv_files, copy_kv_file, delete_kv_file
+
+# Search for KV files in a directory
+kv_files = search_kv_files("test")
+print(f"Found KV files: {kv_files}")
+
+# Copy a KV file
+copy_kv_file("test/test.kv", "test/copy_test.kv")
+print("KV file copied.")
+
+# Delete a KV file
+delete_kv_file("test/copy_test.kv")
+print("KV file deleted.")
+```
+
+### KVFileDiffChecker
+```python
+from kvprocessor import KVFileDiffChecker
+
+file1 = "test/file1.kv"
+file2 = "test/file2.kv"
+diff_checker = KVFileDiffChecker(file1, file2)
+differences = diff_checker.diff()
+print(f"Differences between files: {differences}")
+```
+
+### KVValidator
+```python
+from kvprocessor import validate_kv_file
+
+kv_file_path = "test/test.kv"
+is_valid = validate_kv_file(kv_file_path)
+print(f"KV file is valid: {is_valid}")
+```
+
+### Additional Data Types
+The library supports additional data types such as `datetime`, `date`, `time`, and `decimal`. These can be used in `.kv` files as follows:
+
+```custom
+EVENT_DATE<datetime>:none
+PRICE<decimal>:none
+```
+
+Example usage:
+```python
+from kvprocessor import KVProcessor
+
+kv_file_path = "test/test.kv"  # Path to your .kv file
+kv_processor = KVProcessor(kv_file_path)
+
+# Example configuration with additional data types
+config = {
+    "EVENT_DATE": "2025-05-01T12:00:00",
+    "PRICE": "19.99",
 }
+
+validated_config = kv_processor.process_config(config)
+print(validated_config)
 ```
 
 ## Building
 For building the library locally \
-**Requires**: `python3.8+`, `pip`, `linux system`(if using the predefined shell files)
+**Requires**: `python3.8+`, `pip`, `linux system` (if using the predefined shell files)
+
+1. `git clone https://github.com/connor33341/kvProcessor.git`
+2. `cd kvProcessor`
+3. `bash build.sh`
+
+`build.sh` will also install kvProcessor as a local package, which you will be able to use. If you add new features to your fork and would like them to be featured on the main repo, submit a Pull Request.
+
+## CLI
+At the current moment, there exists no documentation on this. If you would like to find usage, visit the file `kvprocessor\cli.py`. \
+\
+Basic Usage:
+```bash
+python kvprocessor/cli.py --version
+```
 
- 1. `git clone https://github.com/connor33341/kvProcessor.git`
- 2. `cd kvProcessor`
- 3. `bash build.sh`
+## Library Modules
+For a complete list, visit `kvprocessor\__init__.py`. A breif list of main modules, will be listed here.
+ - `kvprocessor.kvprocessor`, Exports: `KVProcessor`
+ - `kvprocessor.kvstructloader`, Exports: `KVStructLoader`
+ - `kvprocessor.kvmanifestloader`, Exports: `KVManifestLoader`
 
-`build.sh` will also install kvProcessor as a local package, which you will be able to use.
-If you add new features to your fork, and would like them to be featured on the main repo. Submit a Pull Request
 ## For the nerds
-The syntax was already mentioned, however if you would like to see how it parses. The following regex is used to determine the: `name`, `type`, and `default`:
+The syntax was already mentioned, however, if you would like to see how it parses, the following regex is used to determine the: `name`, `type`, and `default`:
 ```re
 (\w+)<([\w\|]+)>:([\w+]+|none)
 ```
-With this knowlege, you probably can figure out a way to write .kv files in a weird way. Out of typical standard.
+With this knowledge, you probably can figure out a way to write `.kv` files in a weird way, out of typical standard.

SECURITY.md

@@ -0,0 +1,20 @@
+# Security Policy
+
+## Supported Versions
+
+Versions below `0.1.7` are no longer supproted. `0.2.12`, and `0.2.14-alpha` are both not supproted. Migrate to `0.2.14` w/o the alpha designation, for a stable release.
+
+| Version | Supported          |
+| ------- | ------------------ |
+| 0.2.14+ | :white_check_mark: |
+| 0.2.12  | :x:                |
+| 0.2.1-2 | :x:                |
+| 0.1.10+ | :white_check_mark: |
+| < 0.1.7 | :x:                |
+
+## Reporting a Vulnerability
+
+Contact the developer: <connor@connor33341.dev> \
+\
+Vunerabilites are rare, on this library. However, there is still a potental for them to exist. Please report these, so that they can be delt with accordingly.
+There is no set style inwhich you have to submit a Vunerability. However, I would recomend being detailed, and having the information of what file or library is causing the issue.

TODO.md

@@ -0,0 +1,17 @@
+# TODO
+
+### Documentation:
+ - Create a doc/wiki with mkdocs
+ - Update `README.md` to have basic examples for 0.2.x versions of the API
+
+### Testing:
+ - Update test.py to test for most usecases with the 0.2.x API
+ - Use pytest aswell
+ - Create a GHAction to makesure that all tests are passed.
+
+### API:
+ - Migrate to `kvprocessor.errors` for custom errors
+ - Update cli to support 0.2.x verions of the API
+ - Expand `kvprocessor.kvtypemap` to support more types.
+ - Update `kvprocessor.kvmanifestloader` to be more feature rich
+ - Fix the style, so it will lint correctly

kvprocessor/__init__.py

@@ -1,6 +1,26 @@
-__version__ = "0.1.12"
+__version__ = "0.2.14"
 
 from .kvprocessor import KVProcessor
-from .kvenvloader import LoadEnv
+from .kvenvloader import load_env, LoadEnv
 from .kvstructloader import KVStructLoader
-from .errors import KVProcessorError, InvalidKVFileError, MissingEnvironmentVariableError, NamespaceNotFoundError, InvalidNamespaceError
+from .kvfileexporter import KVFileExporter
+from .kvfilemerger import KVFileMerger
+from .kvfileutils import (
+    search_kv_files,
+    copy_kv_file,
+    delete_kv_file,
+)
+from .kvdiff import KVFileDiffChecker
+from .kvnamespacemanager import NamespaceManager as KVNamespaceManager
+from .kvmanifestloader import KVManifestLoader
+from .kvversionmanager import KVVersionManager
+from .kvdiff import KVFileDiffChecker
+from .kvvalidator import KVFileValidator, validate_kv_file, validate_kv_key, validate_kv_value
+from .kvtypemap import get_type_map, set_type_map, remove_type_map, has_type_map, clear_type_map, add_type_map
+from .kvglobalsettings import set_version, get_version, get_version_tuple, get_version_major, get_version_minor
+from .util.errors import KVProcessorError, InvalidKVFileError, MissingEnvironmentVariableError, NamespaceNotFoundError, InvalidNamespaceError
+from .util.warnings import deprecated as kv_deprecated_warning
+from .util.log import log as kv_log
+
+# CLI
+from .cli import main as kv_cli_main # idealy you dont use this, and use cli directly, however, if you are lazy, just import it, then run kvprocessor.kv_cli_main() to run the CLI