How does an NER model predicts an entity with OOV tokens ?

[vignesh-spericorn]

Hello,

I'm currently trying to implement an NER model for identifying 3 custom entities namely Serial Number, Product Name, Product Quantity.
Product name is generally in English, but the Serial Number will be in various formats like ABC-100, AB100CDEF, 123456-789, etc.
I cleaned the text and annotated a few examples and trained a model using 'en_core_web_lg' model. My doubt is many of the tokens in Serial number are OOV (When i ran spacy debug data it mentioned 24% of my training data doesn't have vectors) but the trained model is able to identify Serial numbers. How is this happening ?

Thanks in advance.

[adrianeboyd]

In the built-in "tok2vec" architectures, the static vectors are not the only feature used, see e.g. MultiHashEmbed: https://spacy.io/api/architectures#MultiHashEmbed