Files
doczyai-pipelines/fieldExtraction/crosswalk
Alex Galarce 53b38ea15c Merged in refactor/split-investment-and-client (pull request #347)
Refactor/split investment and client

* refactor: clean up tests

* Renamed tests/qcqa to tests/qa_qc for consistency

* comment out faulty test - see docstring note at top

* split to client/investment

* add __init__.py to investment and client to avoid mypy confusion

* fix imports

* update imports for consistency across investment module

* move back to one config

* try changing import to relative

* add init.py to src

* try relative import

* try one more sys.path.append

* fix path for client main

* fix imports in file_processing

* make prompts common

* move keywords out to `src`

* move smart_chunking_funcs to `src`

* fix mypy error

* start moving to explicit imports

* remove old keywords and fix imports for keywords and prompts

* change all imports to full paths

* fix unit tests

* add readme for new way of running things

* isort after all that import work

* remove unused sys.path.append from some tests


Approved-by: Katon Minhas
2025-01-08 21:59:47 +00:00
..

This framework allows flexible serialization and application of crosswalk tables.

Features

  • Create crosswalks from Excel files or dictionaries
  • Save crosswalks as JSON for easy reuse
  • Apply crosswalks with customizable default behavior
  • Metadata tracking for column mappings

Creating a Crosswalk

from crosswalk_utils import CrosswalkBuilder

# From Excel
builder = CrosswalkBuilder()
builder.from_excel(
    path="CROSSWALK_PROVIDER_TYPE.20220106.xlsx",
    sheet_name='Prac',
    from_col="Practitioner Specialty",
    to_col="Taxonomy"
).save('crosswalk_provider_type.json')

# You can also create from a dictionary
builder.from_dict(
    mapping={'Acupuncturist': '171100000X'},
    from_col='Specialty',
    to_col='Taxonomy'
)

Applying a Crosswalk

import json
from crosswalk_utils import apply_crosswalk

# Load the mapping
with open('crosswalk_provider_type.json') as f:
    data = json.load(f)
    
# Apply the crosswalk
result = apply_crosswalk(
    val="Acupuncturist",
    mapping=data['mapping'],
    default=None  # Optional: specify default for unmapped values
)
# Result: '171100000X'

Reversing a Crosswalk

from crosswalk_utils import CrosswalkBuilder
builder = CrosswalkBuilder()
builder.from_json("mappings/crosswalk_program_lob.json").create_reverse_mapping()

# Result: 
# {'Commercial-Group': 'Administrative Services Only (ASO), Third-Party '
#                      'Administrator (TPA), Self-Funded, Employer Funded',
#  'Marketplace-Exchange-Individual & Family': 'Platinum, Gold, Silver, Bronze',
#  'Medicaid': 'Aged, Blind, Disabled (ABD), Childrens Health Insurance Program '
#              '(CHIP), Childrens Health Insurance Program Perinate '
#              '(CHIP-Perinate), Foster Care, Intellectual and Developmental '
#              'Disability (IDD), Temporary Assistance Needy Families (TANF)',
#  'Medicare': 'Medicare Advantage, Supplemental',
#  'Medicare-Medicaid': 'Chronic Condition Special Needs Plan (C-SNP), Dual '
#                       'Eligible Special Needs Plan (D-SNP), Institutional '
#                       'Special Needs Plan (I-SNP), Long-Term Services and '
#                       'Supports (LTSS), Program of All-Inclusive Care for the '
#                       'Elderly (PACE), Special Needs Plan (SNP)'}