[Question]: Seeking Feedback on Using LLMs for Hierarchical Multi-Label Text Classification on Large Archive Files (warc)

[reversingentropy]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

I'm new to LLMs and would like feedback on using them for hierarchical multi-label text classification on extensive archive files (.warc). I have training abstracts with over 1000 target categories, and each abstract can have up to 10 labels.

Plan

1. Create embeddings for the warc files or create an abstract.
2. Create embeddings for all 1000 hierarchical target classes.
3. Use Retrieval Augmented Generation to prompt the LLM to select up to 10 labels based on embeddings
4. Fine-tune the model with my training data.

Question

1. Is my proposed workflow realistic?
2. Can I fine-tune either the model or the embeddings?
3. Which is better for my project: LLama or ChatGPT?
4. Can I train the LLM by supplying both the abstract and target category classes within the prompt?
5. Are agents more effective?
6. Would using an agent for recursive category selection from root to leaf nodes be more realistic? Is it worth exploring agents?

Thank you!

[dosubot[bot]]

🤖

Hi there,

Thanks for reaching out with your questions. Your plan seems well thought out and it's great to see you're considering using LLMs for such a complex task. Let's go through your questions:

1. Your proposed workflow seems realistic, but the success will largely depend on the quality of your data and the specifics of your implementation.

2. Yes, you can fine-tune both the model and the embeddings. Fine-tuning the model involves adjusting the weights of the model based on your specific task and data. Fine-tuning the embeddings involves adjusting the representations of the words/phrases in your data to better suit your task.

3. The choice between LLama and ChatGPT would depend on the specifics of your task. Both models have their strengths and weaknesses. LLama is designed for tasks that involve structured data, while ChatGPT is designed for conversational tasks.

4. Yes, you can train the LLM by supplying both the abstract and target category classes within the prompt. This is a common approach in multi-label classification tasks.

5. The effectiveness of agents would depend on the specifics of your task. Agents can be very effective for tasks that involve complex decision-making or tasks that require the model to maintain a state.

6. Using an agent for recursive category selection from root to leaf nodes could be a good approach, especially if your categories are hierarchical. However, this would likely require a more complex implementation and more computational resources.

To provide more specific advice, it would be helpful to know more about your data and your task. For example, what is the nature of your hierarchical target classes? How are your .warc files structured? What is the size of your training data?

Looking forward to hearing more about your project.

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.