Performance Profiling Report

Date: 2025-12-06

Summary

Comprehensive performance profiling of DSPU modules to identify optimization opportunities.

Profiled Modules

Module	Status	Time (s)	Total Calls
I/O Operations	❌ Error	0.222	N/A
Config Loading	❌ Error	0.087	N/A
Validation & Filtering	✅ Success	0.087	103,265
ML Utilities	✅ Success	0.416	1,608,495
Async Utilities	✅ Success	1.008	15,290

---

Detailed Results

I/O Operations

Status: ❌ Error

Error: Cannot detect format from path: file_0.txt Suggestion: Specify format explicitly or use known extensions: .json, .msgpack, .msgpk, .mp, .pkl, .pickle

---

Config Loading

Status: ❌ Error

Error: Config dependencies not available: No module named 'dotenv'

---

Validation & Filtering

Status: ✅ Success Execution Time: 0.087s Total Function Calls: 103,265

Top Functions by Cumulative Time

103265 function calls (102703 primitive calls) in 0.086 seconds

   Ordered by: cumulative time
   List reduced from 797 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.087    0.087 /Users/228496/research/dspu/scripts/profile_performance.py:82(profile_validation)
     25/1    0.000    0.000    0.078    0.078 <frozen importlib._bootstrap>:1349(_find_and_load)
     24/1    0.000    0.000    0.078    0.078 <frozen importlib._bootstrap>:1304(_find_and_load_unlocked)
     24/1    0.000    0.000    0.078    0.078 <frozen importlib._bootstrap>:911(_load_unlocked)
     24/1    0.000    0.000    0.078    0.078 <frozen importlib._bootstrap_external>:993(exec_module)
     51/2    0.000    0.000    0.078    0.039 <frozen importlib._bootstrap>:480(_call_with_frames_removed)
    150/1    0.005    0.000    0.078    0.078 {built-in method builtins.exec}
        1    0.000    0.000    0.078    0.078 /Users/228496/research/dspu/src/dspu/validation/__init__.py:1(<module>)
   125/11    0.001    0.000    0.076    0.007 {built-in method builtins.__build_class__}
        1    0.000    0.000    0.076    0.076 /Users/228496/research/dspu/src/dspu/validation/validators.py:1(<module>)

---

ML Utilities

Status: ✅ Success Execution Time: 0.416s Total Function Calls: 1,608,495

Top Functions by Cumulative Time

1608495 function calls (1608453 primitive calls) in 0.416 seconds

   Ordered by: cumulative time
   List reduced from 178 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.009    0.009    0.416    0.416 /Users/228496/research/dspu/scripts/profile_performance.py:98(profile_ml_operations)
      200    0.158    0.001    0.222    0.001 /Users/228496/research/dspu/src/dspu/ml/scaling.py:177(transform)
      100    0.080    0.001    0.112    0.001 /Users/228496/research/dspu/src/dspu/ml/scaling.py:269(inverse_transform)
  1000049    0.086    0.000    0.086    0.000 {method 'append' of 'list' objects}
      100    0.021    0.000    0.038    0.000 /Users/228496/research/dspu/src/dspu/ml/encoding.py:139(transform)
      404    0.011    0.000    0.030    0.000 {built-in method builtins.sum}
       50    0.000    0.000    0.023    0.000 /Users/228496/research/dspu/src/dspu/ml/stats.py:111(correlation)
       50    0.000    0.000    0.023    0.000 /Users/228496/research/dspu/src/dspu/ml/stats.py:152(_pearson_correlation)
   301803    0.017    0.000    0.017    0.000 {built-in method builtins.len}
   100000    0.009    0.000    0.009    0.000 /Users/228496/research/dspu/src/dspu/ml/encoding.py:188(_encode_label)

---

Async Utilities

Status: ✅ Success Execution Time: 1.008s Total Function Calls: 15,290

Top Functions by Cumulative Time

15290 function calls (15216 primitive calls) in 1.008 seconds

   Ordered by: cumulative time
   List reduced from 347 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    1.008    1.008 /Users/228496/research/dspu/scripts/profile_performance.py:128(profile_async_utilities)
        1    0.000    0.000    1.002    1.002 /Users/228496/.pyenv/versions/3.12.10/lib/python3.12/asyncio/runners.py:160(run)
        3    0.000    0.000    1.002    0.334 /Users/228496/.pyenv/versions/3.12.10/lib/python3.12/asyncio/base_events.py:655(run_until_complete)
        3    0.000    0.000    1.002    0.334 /Users/228496/.pyenv/versions/3.12.10/lib/python3.12/asyncio/base_events.py:631(run_forever)
        1    0.000    0.000    1.002    1.002 /Users/228496/.pyenv/versions/3.12.10/lib/python3.12/asyncio/runners.py:86(run)
      206    0.001    0.000    1.002    0.005 /Users/228496/.pyenv/versions/3.12.10/lib/python3.12/asyncio/base_events.py:1922(_run_once)
      206    0.000    0.000    0.996    0.005 /Users/228496/.pyenv/versions/3.12.10/lib/python3.12/selectors.py:558(select)
      208    0.996    0.005    0.996    0.005 {method 'control' of 'select.kqueue' objects}
      9/1    0.000    0.000    0.007    0.007 <frozen importlib._bootstrap>:1349(_find_and_load)
      9/1    0.000    0.000    0.006    0.006 <frozen importlib._bootstrap>:1304(_find_and_load_unlocked)

---

Performance Insights

Key Observations

• Fastest Module: Validation & Filtering (0.087s)
• Slowest Module: Async Utilities (1.008s)

Optimization Opportunities

Review the detailed profiling data above to identify: - Functions with high cumulative time (potential bottlenecks) - Functions called frequently (optimization candidates) - Import overhead in hot paths

---

Next Steps

1. Hot Path Analysis: Focus on functions with highest cumulative time
2. Call Count Reduction: Optimize frequently called functions
3. Caching: Consider caching for expensive operations
4. Lazy Loading: Defer imports of optional dependencies