CSV Transformer Pro is a powerful Python tool for processing, transforming, and analyzing CSV files. It offers a wide range of features including column transposition, data aggregation, anomaly detection, and advanced validation.
- Read and parse CSV files with support for different delimiters and quoted fields
- Transform CSV data by transposing specified columns
- Infer column data types automatically
- Apply aggregation functions to grouped data
- Perform window function operations on specified columns
- Detect anomalies using various statistical methods
- Validate data using customizable rules
- Generate advanced reports with data visualizations
- Docker
- Docker Compose
-
Clone this repository:
git clone https://github.com/yourusername/csv-transformer-pro.git cd csv-transformer-pro
-
Build the Docker image:
docker-compose build
Run the script using Docker Compose:
docker-compose up
You can also run the script with specific options by creating a docker-compose.override.yml
file with the desired command:
version: '3'
services:
csv-transformer:
command: python main.py input_data.csv -o output_data.csv -t Date,Value -a Currency -w ValueDate1 -d zscore -v Email,Phone
Then, run the script with:
docker-compose up
For more advanced usage and customization options, refer to the inline comments in the script and the provided test suite.
The script includes error handling to catch and provide informative messages for common errors that may occur during the transformation process. This includes missing columns, failed operations, and invalid input options. Any unexpected errors are also caught and raised with a general error message.
The script includes a comprehensive test suite to ensure the correct functioning of its key components. Run the tests using the following command:
docker-compose exec csv-transformer python -m unittest test_csv_transformer.py
Ensure that all tests pass before using the script on real data. If a test fails, it indicates a potential issue with the corresponding part of the script.
This project is licensed under the MIT License - see the LICENSE file for details.
This script is provided under the MIT License. While efforts have been made to ensure its functionality, it is provided "as is", without warranty of any kind. The authors or copyright holders shall not be liable for any claim, damages, or other liability arising from, out of, or in connection with the software or the use or other dealings in the software. Please use at your own risk and review the LICENSE file for more details.