Docling <page_assemble_model> reading order algorithm

[mllife]

Question

...

I see the flow is detecting page layout using the rtdetr model, which have label classes from doclaynet along with some additional labels, then use tableformer for table structure, but what exactly you do for reading order? I see some clustering mentioned in the code for page assemble? Can you explain the code and add some example to explain it? Any test or sample code I can follow to understand this better?

[mllife]

@PeterStaar-IBM , can you help with this?

[mllife]

@trinanjan12 , will you keep exploring this further? i am interested in knowing the algo

[trinanjan12]

@mllife I don't have plans for digging into deep search right now. Mostly I'll check some other reading order models like layoutreader

[mllife]

@trinanjan12 but layoutreader current implemention on github is based on layoutlmv3 which is licence restricted and only way to get around that is training LiLT on readingbank dataset; i have dig a lot deep into this but nothing practical so far

[trinanjan12]

@mllife someone.referred me to this

https://github.com/ppaanngggg/layoutreader. Will check that. I'm not sure about the license though

[mllife]

@mllife @trinanjan12 To shed some light on this topic:

• Neither the old nor the new layout postprocessing do the final reading order, they only prepare the clusters in a "reliable" order.
• The actual reading-order is changed entirely in this unit. The current implementation is in C++ inside the deepsearch-glm package. We have plans to pull it out of there and make it more transparent in the future.

@cau-git , any eta on this? is this what you are refering to https://github.com/DS4SD/docling-ibm-models/pull/44/files#diff-5035f6fb41f8e0a341e1cc96de528f0b496e1697d0952b2d3145a2eb99b74ab1

any source for reading more about this algorithm?