Alex Galarce
0562b90646
io_utils documentation * add docstring to read_local function and handle file read errors gracefully by returning None on failure * update read_local function to support .xlsx files and improve error handling * add docstring to read_s3 function to clarify its purpose and return value * refactor read_s3 and read_s3_csv functions to improve error handling and add type hints * improve error handling in read_s3_csv function for CSV file parsing * enhance read_input function with error handling and detailed docstring * Add TODOs for read_input_csv and filter_already_processed functions; add type hints and docstring for remove_txt_extension * enhance write_local function documentation; make docstrings more consistent when there is a "notes" section * enhance write_local and write_s3 functions; improve documentation, add error output type handling, and ensure consistent CSV writing options * refactor test_io_utils.py; reorganize imports and enhance read_input_local test with additional mock for file existence Approved-by: Katon Minhas
Field Extraction
Running the Code
This project uses Python's module system for imports. To run the code properly:
-
DO NOT run files directly like this:
python src/client/main.py # ❌ This won't work -
INSTEAD, use Python's module flag (
-m) from the project root:python -m src.client.main # ✅ This is correct
Why?
The code uses absolute imports (e.g., from src.utils import llm_utils) to maintain a clear and consistent package structure. Running with python -m ensures Python can properly resolve these imports.
Development
- All imports should use the
src.prefix (e.g.,from src.utils import llm_utils) - Always run code from the project root directory using the
-mflag - Tests are configured to handle these imports automatically via pytest settings in pyproject.toml
Configuration
The project uses poetry for dependency management and pytest for testing. Key configurations in pyproject.toml:
[tool.pytest.ini_options]
pythonpath=["."]
testpaths = [
"tests"
]
Dependencies
To install dependencies:
poetry install
To run tests:
poetry run pytest
To run type checking:
poetry run mypy .