In fact, eight of the nine happened outside their 90% confidence interval, meaning the experts thought there was less than a 10% chance they would happen as early as they did!īut actually it’s much worse than that. This time, nine events happened earlier than the experts thought, and zero happened later, or on time. You can never step in the same river twice, but this survey tried hard to perfectly match its predecessor. Most of the experts were new, but about 6% (45 out of 740) were repeats from the previous round. The new asked the same questions with the same wording. The new version used the same definition of “expert” - a researcher who had published at the prestigious NeurIPS or ICML conferences - and got about the same response rate (17% in 2022 compared to 21% in 2016). As for Katja, her AI forecasting project grew into a seven-person team, with its monthly dinners becoming a nexus of the Bay Area AI scene. The science of forecasting, which only reached public attention after the publication of Philip Tetlock’s Superforecasting in late 2015, took off, and started being integrated into government decision-making. AI got billions of dollars in venture capital investment, spearheaded by fast-growing startup OpenAI and its superstar GPT and DALL-E models. The six years between 20 were good ones for AI, forecasting, and Katja. It’s hard and subjective to figure out exactly when AI first achieved something, but grading the list as best I can: In what year did they think it was as likely as not that AI would reach the milestone? In what year did they think there was even a 10% chance AI would reach it? A 90% chance? What did they think was the chance AI would reach the milestone by 2026? By 2036? I focus on their median prediction of when AI will reach the milestone. Experts were asked to predict the milestones in several ways. The survey asked about 32 specific milestones. Looking at the most zoomed-out summary - whether they underestimated progress, over-hyped it, or got it just right - it’s hard to come to any conclusion other than “just right.” How did the dazzling reality compare to what experts had predicted on Grace’s survey six years earlier? Media called the sudden rise of ChatGPT " shocking," breathtaking," and " mind-blowing." I wondered how it looked from inside the field. Last year, AI started writing high school essays (laundry folding and Angry Birds remain unconquered). Three hundred fifty-two of them took time out of their busy schedules to answer, producing a unique time capsule of expert opinion on the cusp of the AI revolution. The world’s leading AI scientists are a surprisingly accommodating group. When will AI be able to fold laundry? Write high school essays? Beat humans at Angry Birds? Why doesn’t the public understand AI? Will AI be good or bad for the world? Will it kill all humans? In 2016, three years before OpenAI released GPT-2 and the world went crazy, a researcher named Katja Grace cold-emailed the world’s leading AI scientists.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |