article thumbnail

Rising Tide Rents and Robber Baron Rents

O'Reilly

By the end of 2012, it was up to 82%. They published the original Transformer paper (not quite coincidentally called “Attention is All You Need”) in 2017, and released BERT , an open source implementation, in late 2018, but they never went so far as to build and release anything like OpenAI’s GPT line of services.

article thumbnail

Structural Evolutions in Data

O'Reilly

” There’s as much Keras, TensorFlow, and Torch today as there was Hadoop back in 2010-2012. And a quick survey of agent-based modeling and evolutionary algorithms turns up a mix of proprietary apps and nascent open-source projects, some of which are geared for a particular problem domain.

article thumbnail

Real-Real-World Programming with ChatGPT

O'Reilly

To provide some coherence to the music, I decided to use Taylor Swift songs since her discography covers the time span of most papers that I typically read: Her main albums were released in 2006, 2008, 2010, 2012, 2014, 2017, 2019, 2020, and 2022. This choice also inspired me to call my project Swift Papers.