ADM-201 dump PMP dumps pdf SSCP exam materials CBAP exam sample questions

OpenAI GPT-2:几乎完美的文本生成器 – 译学馆
未登录,请登录后再发表信息
最新评论 (0)
播放视频

OpenAI GPT-2:几乎完美的文本生成器

OpenAI GPT-2: An Almost Too Good Text Generator

Dear Fellow Scholars, this is 2 minutes paper Károly Zsolnai-Fehér
学者朋友们大家好 这里是两分钟论文
This is an incredible paper from OpenAI
这是Open Al最近发表的一篇超棒的论文
in which the goal is to teach AI to read a piece of text
论文的目的是 让人工智能学习文本阅读
and perform common nature language processing operation.
并进行常规的自然语言处理应用
For instance, answering questions, completing text,
例如 回答问题 文本补充
reading comprehension, summarization, and more.
阅读理解 归纳总结等等
And not only that,
不仅如此
but additionally, the AI has to be able to perform these tasks
人工智能还需要在尽可能少的监督下
with as little supervision as possible.
处理这些任务
This means that we seek to unleash the algorithm that they call GPT-2
这就意味着我们需要运用一种叫GPT-2的算法
to read the internet and learn the intricacies of our language by itself.
来读取互联网 并主动学习我们错综复杂的语言
To perform this, of course, we need a lot of training data
当然 为了达到目的 我们需要大量的训练数据
and here, the AI reads 40 gigabytes of internet text,
文章里的人工智能已经读取了400亿字节的互联网文本
which is 40 gigs of non-binary plaintext data,
包括40GB的非二进制纯文本数据
which is a stupendously large amount of text.
这是极大的文本量
It is always hard to put these big numbers in context,
将如此大量的文本融入到具体语境是很困难的
so as an example, to train similar text completion algorithms,
举个例子 为了训练类似的文本处理算法
AI people typically reach out to a text file
人工智能学者通常会用一个
containing every significant work of Shakespeare himself
包含莎士比亚所有经典著作的文件
and this file is approximately 5 megabytes.
文件大约有5MB
so the 40 gigabytes basically means an amount of text
所以40GB相当于
that is 8000 times the size of Shakespeare’s works.
莎士比亚作品集大小的8000倍
That’s a lot of text
这是非常大的文本量
And now, let’s have a look at how it fares with the text completion part.
现在 让我们看看它是如何进行文本补充的
This part was written by a human,quoting:
这部分由人类撰写 内容是:
“In a shocking finding,
“震惊
scientist discovered a herd of unicorns living in a remote
科学家发现了生活在安第斯山脉
previously unexplored valley, in the Andes Mountains.
人迹罕至的峡谷的一群独角兽
even more surprising to researchers was the fact that the unicorns spoke perfect English. ”
更令科学家震惊的是 独角兽说着非常流利的英语”
And the AI continued the text the following way, quoting a short snippet of it:
然后由人工智能续写 其中的一小段是
“The scientist named the population, after their distinctive horn, Ovid’s Unicorn.
“科学家根据它们独特的角 将其命名为奥维德独角兽
These four-horned, silver-white unicorns were previously unknown to science.”
这是科学界第一次发现银白色的四角独角兽”
Wow! Now note that this is clearly not perfect.
哇!请注意目前的续文并不完美
if there is even such a thing as a perfect continuation, and it took 10 tries,
如果一个所谓的完美续文需要十次尝试
which means that the algorithm was run 10 times
意味着这个算法会运行十次
and the best result was cherrypicked and recorded here.
然后最优解就会被挑选出并记录下来
And despite all of these,
尽管这些已经很棒
this is a truly incredible result,
更不可思议的是
especially given that the algorithm learns on its own.
这个算法能自己学习
After giving it a piece of text,
在给它一篇文章后
it can also answer questions in a quite competent manner.
人工智能也能按要求回答问题
Worry not, later in this video,
别着急 接下来的视频中
I will show you more of these examples and likely talk over them
我将向您展示更多的相关例子 并尽量详尽地解释
so if you are curious,
所以如果您好奇的话
feel free to pause the video
在阅读提示和文本补全时
while you read the prompts and their completions.
可以随时暂停本视频
The validation part of the paper reveals
论文的验证部分发现
that this method is able to achieve state-of-the-art results
这个算法能够在几种语言的建模任务上
on several language modeling tasks,
达到最先进的水平
and you can see here
你们可以看到这里
that we still shouldn’t expect it to match a human in terms of reading comprehension,
就阅读理解这种问答测试而言
which is the question answering test.
我们不应期望人工智能比肩人类
More on that in a moment.
稍后会做详细介绍
So, there are plenty of natural language processing algorithms out there
还有许多的自然语言处理算法
that can perform some of these tasks,
可以完成部分上述任务
in fact, some articles already stated that there is not much new here,
实际上许多文章都表示这并不新颖
it’s just the same problem,
只是用更普遍的方法
but stated in a more general manner, and with more compute.
和更复杂的计算解决同一个问题
A ha! It is not the first time that this happens.
啊哈 不是第一次出现这个情况了
Remember our video by the name “The Bitter lesson”?
还记得我们名为《沉痛教训》的视频吗
I’ve put a link to it in the video description,
我在视频描述里放上了视频链接
but in case you missed it,
但以防你没看到
let me quote how Richard Sutton addressed his situation:
让我来引用理查德·萨顿对这种情况的表述
“The bitter lesson is based on the historical observations
“苦痛的教训基于历史观察发现
that 1 ) AI researchers have often tried to build knowledge into their agents,
包括1)AI研究者经常尝试在算法中创建自己的知识体系
2 ) this always helps in the short term
2)这种方法通常在短期内
and is personally satisfying to the researcher,
达到让研究者满意的结果
but 3 ) in the long run it plateaus
但是 3)长期则效果不明显
and even inhibits further progress
甚至影响更深入的进展
and 4 ) breakthrough progress eventually arrives by an opposing approach
4)最终的突破往往来自于相反的
based on scaling computation by search and learning.
基于探索和学习的标度计算方法
The eventual success is tinged with bitterness,
最终的成功 略带苦涩
and often incompletely digested,
且常常不被完全接受
because it success over a favored, human-centric approach.“
因为它的成功颠覆了以人类为中心的主流方法”
So what is the big lesson here?
那么这里的重点是什么呢?
Why is GPT-2 so interesting?
为什么GPT-2如此有趣?
Well, big lesson number one is that
第一点
this is one of the clearer cases of what the quote was talking about,
也是引言中最明显的一点
where we can do a whole lot given a lot of data and compute power,
假设我们有大量数据和运算能力 可以完成很多任务
and we don’t need to insert too much additional knowledge into our algorithms.
且并不需要在算法中加入太多额外的内容
And lesson number two,
第二点是
as a result, this algorithm becomes quite general
这个算法因此变得具有普适性
So it can perform more tasks than most other techniques.
比起其他技术 它能完成更多的任务
This is an amazing value proposition.
这是一个很棒的价值主张
I will also add
我还想提出来
that not every learning technique scales well when we add more compute,
算力提升 并不能有效提升每种学习技术的适应性
in fact,you can see here yourself
事实上 你可以看到
that even GPT-2 plateaus on the summarization task.
即使是GPT-2也会在归纳任务上受阻
Making sure that these learning algorithms scale well
保证这些学习算法的适应性
is a great contribution in and of itself
本身就是一个巨大的贡献
and should not be taken for granted.
不应该被轻视
There has been a fair bit of discussion on
目前已经有很多关于OpenAI
whether OpenAI should publish the entirety of this model.
是否应该公开完整模型的讨论
They opted to release a smaller part of the source code
最终 他们选择发布源代码的一小部分
and noted that they are aware
注意他们确实意识到
that the full model could be used for nefarious purposes.
一些人可能出于非法目的利用完整的模型
Why did they do this?
OpenAI为什么这样做
What is the matter with everyone having an AI
每个人都拥有一个仅次于人类阅读理解水平的
with a subhuman-level reading comprehension?
人工智能助手有什么问题吗?
Well, so far, we have only talked about quality.
目前我们只讨论了性能
But another key part is quantity.
但另一个关键部分是数量
And boy, are these learning methods superhuman in terms of quantity
这些学习方法在数量方面是超乎常人的吗?
just imagine that they can write articles
想象一下 它们可以
with a chosen topic and sentiment all day long.
根据设定的主题和情感创作一整天
and much quicker than human beings
速度比人类还快
Also note that the blueprint of the algorithm is described in the paper,
还请注意作者在论文中描绘了算法的蓝图
and a top-tier research group is expected to be able to reproduce it.
期望顶尖的研究团队能将其再现
So does one release the full source code and models or not?
这算不算泄露了完整的源代码和模型呢?
This is a quite difficult question:
这是一个很难界定的问题:
we need to keep publishing both papers and source code to advance science,
推动科学的发展需要论文和源代码的不断发表
but we also have to find new ways to do it
但我们必须在具备职业道德的手段下
in an ethical manner.
寻找新的实现方法
This needs more discussion
这需要进一步讨论
and would definitely be worthy of a conference-style meeting, or more.
值得召开一个或多个专题研讨会
There is so much to talk about,
有太多需要讨论的问题
and so far,we have really only scratched the surface,
目前 我们也只是接触到了皮毛
So make sure to have a look in the video description,
所以请您务必看一下视频简介
I left a link to the paper
我附上了这篇论文的链接
and some more super interesting reading materials for you.
和一些非常有趣的阅读材料
Make sure to check them out.
记得查看
Also just a quick comment on
另外 快速解释一下
why this video came so late after the paper has appeared.
为什么本视频在论文发表这么久后才发布
Since there were a lot of feelings and intense discussion on
因为有许多反应和激烈的争论
whether the algorithm should be published or not,
聚焦于是否应该公布该算法
I was looking to wait until the dust settles
我一直在等尘埃落定后
and there is enough information out there
有足够的信息
to create a sufficiently informed video for you.
来为你们制作一个更全面的视频
This of course means that we are late to the party
当然 这就意味着我们错过了热度
and missed out on the whole lot of views and revenue.
并且失去了许多播放量和收入
But that’s okay.
但是没关系
In fact, that’s what we’ll keep doing going forward
事实上 我们长久以来的宗旨是
to make sure you get the highest quality information that I can provide.
为观众朋友们提供最优质的信息
If you have enjoyed this episode
如果你们喜欢这期节目
and would like to help us,
并且愿意帮助我们
please consider supporting us on Patreon.
请考虑在Patreon上支持我们
Remember our motto,
记住我们的座右铭
a dollar a month is almost nothing,
一个月一美元算不得什么
but it keeps the papers coming.
但是它可以让论文持续更新
And there are hundreds of papers on my reading list.
而我的阅读清单有几百篇论文
As always, we are available through Patreon.com/TwoMinutePapers,
老规矩 点击Patreon.com/TwoMinutePapers即可送出支持
and the link is also available in the video description.
你也可以在视频简介里找到链接
Thanks for watching and for your generous support,
感谢您的观看和支持
and see you next time
我们下期再见!

发表评论

译制信息
视频概述

一种名为GPT-2的算法 让人工智能学会文本处理!

听录译者

收集自网络

翻译译者

Germanotta

审核员

审核员1024

视频来源

https://www.youtube.com/watch?v=8ypnLjwpzK8

相关推荐