Learning to summarize with human feedback

• Language
• Generative Models
• Transformers

Footnotes

References

Authors

Nisan Stiennon, Paul Christiano, Daniel Ziegler, Ryan Lowe, Jeffrey Wu, Chelsea Voss, Long Ouyang

Acknowledgments

We’d like to thank the following people who gave feedback on various iterations of the blog post: Douwe Kiela, Zach Lipton, Alex Irpan, Jack Clark, Jacob Hilton, Raul Puri, Miles Brundage, Greg Brockman, Ilya Sutskever, Kelly Sims, Wojciech Kryscinski, and Dzimitry Bahdanau. We’d also like to thank Justin Jay Wang for driving the blog post design, Ashley Pilipiszyn for editing, Alec Radford and Dario Amodei for guidance on the project, Shan Carter for help designing the main diagram, Gretchen Krueger for co-writing the model card, Beth Barnes for help with labeler hiring and general encouragement, and many other people at OpenAI for training our large pre-trained models, supporting us through computing infrastructure improvements and maintenance, and writing fast GPU kernels. Finally, we’d like to thank all of our contractors for providing the data that was essential for training the models in this post.