Improving Table Performance

[brianjking]

Initial Checks

• I confirm that I'm on the latest version

Description

I'm trying to use the https://filimoa.github.io/open-parse/processing/parsing-tables/unitable/ support to extract content out of a UB-04 document - I added !pip install "openparse[ml]" and !openparse-download to the notebook, but I'm not sure what else is required.

Thanks!

Example Code

No response

[Filimoa]

Could you provide the code you're running?

[brianjking]

@Filimoa Sorry, it's your notebook file which I added to.

I'm simply trying to identify the best way to extract text using OpenParse from a document like this:

UB04_empty.pdf

# -*- coding: utf-8 -*-
"""unitable.ipynb

Automatically generated by Colab.

Original file is located at
    https://colab.research.google.com/drive/1Sl-RQTv7Kw_2c2ymr8W_Fpz_HWKdK3v4

## Unitable

When table detection works this yields perfect results for incredibly challenging tables. Unfortunately we still need to use table-transformers for bounding box detection and it's performance leaves something to be desired.

I spoke with the UniTable team and they might implement this. Their challenge lies in PubTables-1M having poor groundtruth annotations.

If you're aware of a better table detection model, please let us know - theoretically this task should be significantly easier than content-extraction.

## Notebook

This notebook demonstrates using unitable to extract some challenging tables we've seen. This is meant to really push the limits of the model.

You will need to use git LFS to download the sample data.
"""

!pip install "openparse[ml]"

!openparse-download

import sys
from pathlib import Path

sys.path.append("..")

import openparse

pdfs_with_tables_dir = Path("/content/pdf")

for pdf_path in pdfs_with_tables_dir.glob("*"):
    parser = openparse.DocumentParser(
        table_args={
            "parsing_algorithm": "unitable",
            "min_table_confidence": 0.8,
        },
        processing_pipeline=None,
    )
    parsed_nodes = parser.parse(pdf_path)
    table_nodes = [node for node in parsed_nodes.nodes if "table" in node.variant]

    if not table_nodes:
        print(f"Could not find tables on {pdf_path}")
        continue

    doc = openparse.Pdf(file=pdf_path)
    doc.display_with_bboxes(table_nodes)

[Filimoa]

Thanks, I'll try running this myself soon and add it to the eval suite. If I had to guess the table-transformers detection is performing poorly (unitable does content extraction, table-transformers still does table bbox detection). This is especially a problem on massive full-page tables like this.

[brianjking]

@Filimoa So just to be clear, that notebook code I posted should have done the extraction similar to the quickstart and is the SOTA method open parse supports, right?

[brianjking]

I added to the quickstart and it tells me there are no tables in the file - which is clearly wrong.

import sys
from pathlib import Path

sys.path.append("..")

import openparse

pdfs_with_tables_dir = Path("/content/pdf")

for pdf_path in pdfs_with_tables_dir.glob("*"):
    parser = openparse.DocumentParser(
        table_args={
            "parsing_algorithm": "unitable",
            "min_table_confidence": 0.8,
        },
        processing_pipeline=None,
    )
    parsed_nodes = parser.parse(pdf_path)
    table_nodes = [node for node in parsed_nodes.nodes if "table" in node.variant]

    if not table_nodes:
        print(f"Could not find tables on {pdf_path}")
        continue

    doc = openparse.Pdf(file=pdf_path)
    doc.display_with_bboxes(table_nodes)
    ```
    

[Filimoa]

I haven't had a chance to test for myself but to clarify yes unitable achieves SOTA performance on converting table images to HTML. Unfortunately it's trained on perfectly cropped tables so we're still forced to rely on table transformers to detect bounding boxes. So if table-transformers doesn't find a table, it won't touch unitable.

I'm actively looking for something with better performance - I haven't had a chance to look into this very deeply, intuitively this seems like a much simpler task than the second stage.

[brianjking]

@Filimoa table-transformers sees the table before this.

[Filimoa]

I'm working on shipping a llama-index integration and then I will spend some time improving table performance - there's a lot of low hanging fruit here.