Synthetic data: save money, time and carbon with open source

tl;dr

Should you fine-tune your own model or use an LLM API? Creating your own model puts you in full control but requires expertise in data collection, training, and deployment. LLM APIs are much easier to use but force you to send your data to a third party and create costly dependencies on LLM providers. This blog post shows how you can combine the convenience of LLMs with the control and efficiency of customized models.

In a case study on identifying investor sentiment in the news, we show how to use an open-source LLM to create synthetic data to train your customized model in a few steps. Our resulting custom RoBERTa model can analyze a large news corpus for around $2.7 compared to $3061 with GPT4; emits around 0.12 kg CO2 compared to very roughly 735 to 1100 kg CO2 with GPT4; with a latency of 0.13 seconds compared to often multiple seconds with GPT4; while performing on par with GPT4 at identifying investor sentiment (both 94% accuracy and 0.94 F1 macro). We provide reusable notebooks, which you can apply to your own use cases.

Table of Contents

• 1. The problem: There is no data for your use-case
• 2. The solution: Synthetic data to teach efficient students
• 3. Case study: Monitoring financial sentiment
  • 3.1 Prompt an LLM to annotate your data
  • 3.2 Compare the open-source model to proprietary models
  • 3.3 Understand and validate your (synthetic) data
  • 3.3 Tune your efficient & specialized model with AutoTrain
  • 3.4 Pros and cons of different approaches
• Conclusion

1. The problem: There is no data for your use-case

2. The solution: Synthetic data to teach efficient students

3. Case study: Monitoring financial sentiment

Imagine you are a developer in a large investment firm tasked with monitoring economic news sentiment toward companies in your investment portfolio. Until recently, you had two main options:

3.1 Prompt an LLM to annotate your data

We start by installing a few required libraries.

We can then download the example dataset with its expert annotations.