Machine learning - Revision history

Lê Nguyên Hoang: /* Empirical observations */

2020-01-27T11:17:17Z

Empirical observations

Lê Nguyên Hoang: /* Turing's argument */

2020-01-27T11:16:45Z

Turing's argument

Lê Nguyên Hoang at 11:09, 27 January 2020

2020-01-27T11:09:36Z

Lê Nguyên Hoang: /* Turing's argument */

2020-01-27T11:08:57Z

Turing's argument

Lê Nguyên Hoang at 11:04, 27 January 2020

2020-01-27T11:04:51Z

Lê Nguyên Hoang: Created page with "Machine learning is the idea of letting algorithms write (or update) their own codes. It is at the heart of recent breakthroughs in computer science, especially in image analy..."

2020-01-20T21:27:09Z

Created page with "Machine learning is the idea of letting algorithms write (or update) their own codes. It is at the heart of recent breakthroughs in computer science, especially in image analy..."

New page

Machine learning is the idea of letting algorithms write (or update) their own codes. It is at the heart of recent breakthroughs in computer science, especially in image analysis, speech recognition, natural language processing, but even problem solving [cite AlphaFold, deep learning for symbolic math]. Perhaps most crucially, there are strong arguments to suggest that it will allow further spectacular progress in the coming years.

== Turing's argument ==

The most compelling case for machine learning is arguably given in Section 7 of [[Turing 1950]] paper.

== Supervised, unsupervised and reinforced ==

There are 3 main forms of learning.

== What makes machine learning safety hard ==

Can't apply formal verification!

== Machine learning is data and goal-driven! ==

This means that we should care about quality data collection, but also on objective function design. It has been argued that the latter should follow the principle of [[alignment]].

@@ Line 19: / Line 19: @@
 <blockquote>More data beats clever algorithms, but better data beats more data. (Peter Norvig)</blockquote>
-<blockquote>When a large language model is trained on a sufficiently large and diverse dataset it is able to perform well across many domains and datasets [...] high-capacity models trained to maximize the likelihood of a sufficiently varied text corpus begin to learn how to perform a surprising amount of tasks without the need for explicit supervision. [https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf RWCLAS][https://scholar.google.ch/scholar?hl=en&as_sdt=0%2C5&q=Language+Models+are+Unsupervised+Multitask+Learners&btnG= 19]</blockquote>
+<blockquote>When a large language model is trained on a sufficiently large and diverse dataset it is able to perform well across many domains and datasets [...] High-capacity models trained to maximize the likelihood of a sufficiently varied text corpus begin to learn how to perform a surprising amount of tasks without the need for explicit supervision. [https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf RWCLAS][https://scholar.google.ch/scholar?hl=en&as_sdt=0%2C5&q=Language+Models+are+Unsupervised+Multitask+Learners&btnG= 19]</blockquote>
 == Supervised, unsupervised and reinforced ==

@@ Line 3: / Line 3: @@
 == Turing's argument ==
-The most compelling case for machine learning is arguably given in Section 7 of [[Turing 1950]] paper.
+The most compelling case for machine learning is arguably given in Section 7 of [[Turing 1950]] paper. Here's a brief version of this argument.
 == Empirical observations ==

← Older revision		Revision as of 11:09, 27 January 2020
Line 4:		Line 4:

	The most compelling case for machine learning is arguably given in Section 7 of [[Turing 1950]] paper.		The most compelling case for machine learning is arguably given in Section 7 of [[Turing 1950]] paper.
		+
		+	== Empirical observations ==

	[http://incompleteideas.net/IncIdeas/BitterLesson.html Sutton19] argues that this is indeed what we have been observing over the last few decades. Cleverly crafted algorithms for numerous tasks have been spectacularly outperformed by machine learning algorithms.		[http://incompleteideas.net/IncIdeas/BitterLesson.html Sutton19] argues that this is indeed what we have been observing over the last few decades. Cleverly crafted algorithms for numerous tasks have been spectacularly outperformed by machine learning algorithms.

@@ Line 5: / Line 5: @@
 The most compelling case for machine learning is arguably given in Section 7 of [[Turing 1950]] paper.
 [http://incompleteideas.net/IncIdeas/BitterLesson.html Sutton19] argues that this is indeed what we have been observing over the last few decades. Cleverly crafted algorithms for numerous tasks have been spectacularly outperformed by machine learning algorithms.
 <blockquote>More data beats clever algorithms, but better data beats more data. (Peter Norvig)</blockquote>
 == Supervised, unsupervised and reinforced ==

← Older revision		Revision as of 11:04, 27 January 2020
Line 4:		Line 4:

	The most compelling case for machine learning is arguably given in Section 7 of [[Turing 1950]] paper.		The most compelling case for machine learning is arguably given in Section 7 of [[Turing 1950]] paper.
		+
		+	[http://incompleteideas.net/IncIdeas/BitterLesson.html Sutton19] argues that this is indeed what we have been observing over the last few decades. Cleverly crafted algorithms for numerous tasks have been spectacularly outperformed by machine learning algorithms.
		+
		+	<blockquote>More data beats clever algorithms, but better data beats more data. (Peter Norvig)</blockquote>

	== Supervised, unsupervised and reinforced ==		== Supervised, unsupervised and reinforced ==