AIOCR

Two years ago, I envisioned building an AI-OCR system, but it remained untouched until now. Having recently completed a custom AI project and found a moment of stability, I decided it was time to bring this AI-OCR project to life.
For this project, rather than building a custom AI model from scratch, I opted to leverage a major, commercially available AI. The primary reason is the overwhelming volume of training data they possess—nothing beats that advantage. However, since no AI can transcribe with absolute perfection, I decided to incorporate specific correction and normalization capabilities.
Now that the foundational text extraction from PDFs and image files is operational, the next phase is to reinforce the secondary extraction step and strictly standardize the output format (JSON). Once a clean JSON file is generated, downstream processes can be seamlessly automated using RPA or other workflow tools.
Key Requirements & Implementation Status
Establishment of a Two-Stage Pipeline (Classification ➔ Extraction)
1st Stage (Classification): The AI determines whether the document is a purchase order and identifies the specific customer via master data integration. If the customer is unidentified, the workflow automatically routes it to a generic fallback rule (UNKNOWN). This flow has been successfully established.
2nd Stage (Extraction): The system dynamically applies customer-specific rules based on the classification result. Testing is underway to validate this mechanism for highly accurate data extraction.
Full Compliance with the Latest Google Official Library (google-genai)
A robust, error-free base code has been finalized, fully conforming to the latest gemini-2.5-flash model and v1 API specifications.
Validation of Multimodal (PDF/Image) Support
The system successfully handles not only digital PDFs but also scanned handwriting or images (PNG, JPG) without code modifications, utilizing an automatic file extension detection logic.
Strict Standardization of Output Format (JSON)
To eliminate inconsistencies between printed text and handwriting, strict prompt constraints are planned to be implemented. This includes forcing the removal of units (e.g., "cases") from quantities to output purely numerical values, and unifying dates into the YYYY/MM/DD format to ensure a rigidly fixed data structure.
System Architecture
【Input File】 (Order.pdf / handwritten.png)
│
▼
┌──────────────────────────────┐
│ File Extension Detection │ ➔ Identifies PDF vs. Image (PNG, etc.)
└──────────────┬───────────────┘ and assigns appropriate labels.
│
▼
┌──────────────────────────────┐
│ [1st Stage] │ ➔ Driven by Gemini-2.5-flash.
│ Classification & Validation │ Determines: "Is it a PO?" and
└──────────────┬───────────────┘ "Which customer (A, B, UNKNOWN)?"
│
├───► [Result] { "is_order": "Yes", "customer_id": "UNKNOWN" }
▼
┌──────────────────────────────┐
│ Master Data Lookup │ ➔ Dynamically retrieves specific prompt
│ (Database Integration) │ rules from CUSTOMER_PROMPT_MASTER
└──────────────┬───────────────┘ based on the detected Customer ID.
│
▼
┌──────────────────────────────┐
│ [2nd Stage] │ ➔ Driven by Gemini-2.5-flash.
│ Data Field Extraction │ Extracts consistent data according to a
└──────────────┬───────────────┘ rigidly defined common format
│ (Destination, Delivery Date, JAN, Qty, Unit).
▼
┌──────────────────────────────┐
│ Output Generation │ ➔ Cleanses the received data to prevent
│ (JSON Export) │ encoding issues and saves it as
└──────────────────────────────┘ "input_filename.json" in the folder.
Next Steps
Final Testing for Handwriting and Printed Text
Using the newly standardized code, actual PDFs and handwritten PNGs will be processed to verify that the output JSON strictly adheres to the exact same structural framework.

Comments