25,161
edits
Changes
Created page with "'''Home * People * Marcus Hutter''' FILE:marcushutter.jpg|border|right|thumb|link=http://www.hutter1.net/index.htm|Marcus Hutter <ref>[http://www.hutter1...."
'''[[Main Page|Home]] * [[People]] * Marcus Hutter'''
[[FILE:marcushutter.jpg|border|right|thumb|link=http://www.hutter1.net/index.htm|Marcus Hutter <ref>[http://www.hutter1.net/index.htm HomePage of Marcus Hutter]</ref> ]]
''Marcus Hutter''',<br/>
a German physicist and computer scientist, professor in the ''Research School of Computer Science'' at [[Australian National University]]. Before, he researched at [https://en.wikipedia.org/wiki/IDSIA IDSIA], [https://en.wikipedia.org/wiki/Lugano Lugano], [https://en.wikipedia.org/wiki/Switzerland Switzerland] in [[Jürgen Schmidhuber|Jürgen Schmidhuber's]] group. Marcus Hutter defended his PhD and BSc in physics from the [https://en.wikipedia.org/wiki/Ludwig_Maximilian_University_of_Munich Ludwig Maximilian University of Munich] and a [https://en.wikipedia.org/wiki/Habilitation Habilitation], MSc, and BSc in computer science from [[Technical University of Munich]]. He is author of the AI-book ''Universal Artificial Intelligence'' <ref>[[Marcus Hutter]] ('''2005'''). ''[http://www.hutter1.net/ai/uaibook.htm Universal Artificial Intelligence]''. Sequential Decisions based on Algorithmic Probability, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]</ref> , a novel algorithmic information theory <ref>[http://www.scholarpedia.org/article/Algorithmic_information_theory Algorithmic information theory - Scholarpedia]</ref> perspective, also introducing the universal algorithmic agent called '''AIXI'''.
=AIXI=
<span id="AIXI"></span>Quote from ''The AIXI Model in One Line'' <ref>[http://www.hutter1.net/ai/uaibook.htm#oneline The AIXI Model in One Line]</ref>
It is actually possible to write down the AIXI model explicitly in one line, although one should not expect to be able to grasp the full meaning and power from this compact representation.
AIXI is an agent that interacts with an environment in cycles k=1,2,...,m. In cycle k, AIXI takes action ak (e.g. a limb movement) based on past perceptions o1 r1...ok-1 rk-1 as defined below. Thereafter, the environment provides a (regular) observation ok (e.g. a camera image) to AIXI and a real-valued reward rk. The reward can be very scarce, e.g. just +1 (-1) for winning (losing) a chess game, and 0 at all other times. Then the next cycle k+1 starts. Given the above, AIXI is defined by:
[[FILE:aixi1line.gif|none|border|text-bottom|link=http://www.hutter1.net/ai/uaibook.htm#oneline]]
The expression shows that AIXI tries to maximize its total future reward rk+...+rm. If the environment is modeled by a deterministic program q, then the future perceptions ...okrk...omrm = U(q,a1..am) can be computed, where U is a universal (monotone Turing) machine executing q given a1..am. Since q is unknown, AIXI has to maximize its expected reward, i.e. average rk+...+rm over all possible perceptions created by all possible environments q. The simpler an environment, the higher is its a-priori contribution 2-l(q), where simplicity is measured by the length l of program q. Since noisy environments are just mixtures of deterministic environments, they are automatically included. The sums in the formula constitute the averaging process. Averaging and maximization have to be performed in chronological order, hence the interleaving of max and Σ (similarly to minimax for games).
=Selected Publications=
<ref>[http://www.hutter1.net/official/publ.htm Publications of Marcus Hutter]</ref> <ref>[http://www.informatik.uni-trier.de/~ley/pers/hd/h/Hutter:Marcus.html dblp: Marcus Hutter:]</ref>
==2005 ...==
* [[Marcus Hutter]] ('''2005'''). ''[http://www.hutter1.net/ai/uaibook.htm Universal Artificial Intelligence]''. Sequential Decisions based on Algorithmic Probability, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Marcus Hutter]] ('''2007'''). ''[http://www.hutter1.net/ai/aixigentle.htm Universal Algorithmic Intelligence: A mathematical top->down approach]''. Technical Report IDSIA-01-03 In Artificial General Intelligence, [http://www.hutter1.net/ai/aixigentle.pdf pdf]
* [[Joel Veness]], [[Kee Siong Ng]], [[Marcus Hutter]], [[David Silver]] ('''2009'''). ''A Monte Carlo AIXI Approximation'', [http://jveness.info/publications/arXive2009%20-%20a%20monte%20carlo%20aixi%20approximation.pdf pdf]
==2010 ...==
* [[Joel Veness]], [[Kee Siong Ng]], [[Marcus Hutter]], [[David Silver]] ('''2010'''). ''Reinforcement Learning via AIXI Approximation''. [[Conferences#AAAI-2010|AAAI-2010]], [http://jveness.info/publications/veness_rl_via_aixi_approx.pdf pdf]
* [[Tor Lattimore]], [[Marcus Hutter]], [http://www.informatik.uni-trier.de/~ley/pers/hd/g/Gavane:Vaibhav.html Vaibhav Gavane] ('''2011'''). ''Universal Prediction of Selected Bits''. [http://www.informatik.uni-trier.de/~ley/db/conf/alt/alt2011.html Algorithmic Learning Theory], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science] 6925, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Tor Lattimore]], [[Marcus Hutter]] ('''2011'''). ''Asymptotically Optimal Agents''. [http://www.informatik.uni-trier.de/~ley/db/conf/alt/alt2011.html Algorithmic Learning Theory], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science] 6925, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Tor Lattimore]], [[Marcus Hutter]] ('''2011'''). ''Time Consistent Discounting''. [http://www.informatik.uni-trier.de/~ley/db/conf/alt/alt2011.html Algorithmic Learning Theory], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science] 6925, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Tor Lattimore]], [[Marcus Hutter]] ('''2011'''). ''No Free Lunch versus Occam's Razor in Supervised Learning''. [https://en.wikipedia.org/wiki/Ray_Solomonoff Solomonoff] Memorial, [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science] 7070, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [https://arxiv.org/abs/1111.3846 arXiv:1111.3846]
* [[Tor Lattimore]], [[Marcus Hutter]] ('''2012'''). ''PAC Bounds for Discounted MDPs''. [http://www.informatik.uni-trier.de/~ley/db/conf/alt/alt2012.htm Algorithmic Learning Theory], [https://arxiv.org/abs/1202.3890 arXiv:1202.3890] <ref>[https://en.wikipedia.org/wiki/Markov_decision_process Markov decision process from Wikipedia]</ref>
* [[Peter Auer]], [[Marcus Hutter]], [[Laurent Orseau]] ('''2013'''). ''[http://drops.dagstuhl.de/opus/volltexte/2013/4340/ Reinforcement Learning]''. [http://dblp.uni-trier.de/db/journals/dagstuhl-reports/dagstuhl-reports3.html#AuerHO13 Dagstuhl Reports, Vol. 3, No. 8], DOI: [http://drops.dagstuhl.de/opus/volltexte/2013/4340/ 10.4230/DagRep.3.8.1], URN: [http://drops.dagstuhl.de/opus/volltexte/2013/4340/ urn:nbn:de:0030-drops-43409]
* [[Tor Lattimore]], [[Marcus Hutter]] ('''2014'''). ''[https://link.springer.com/chapter/10.1007/978-3-319-11662-4_13 Bayesian Reinforcement Learning with Exploration]''. [http://dblp.uni-trier.de/db/conf/alt/alt2014.html Algorithmic Learning Theory], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science] 8776, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
==2015 ...==
* [[Tom Everitt]], [[Marcus Hutter]] ('''2015'''). ''Analytical Results on the BFS vs. DFS Algorithm Selection Problem. Part I: Tree Search''. Australasian Conference on Artificial Intelligence, [https://pdfs.semanticscholar.org/1b4b/c878b2d068214e39b258ee250e5b8889e84c.pdf pdf]
* [[Tom Everitt]], [[Marcus Hutter]] ('''2015'''). ''Analytical Results on the BFS vs. DFS Algorithm Selection Problem: Part II: Graph Search''. Australasian Conference on Artificial Intelligence
* [[Marcus Hutter]] ('''2017'''). ''Universal Learning Theory''. [https://link.springer.com/referencework/10.1007%2F978-1-4899-7687-1 Encyclopedia of Machine Learning and Data Mining 2017], [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
=External Links=
* [https://en.wikipedia.org/wiki/Marcus_Hutter Marcus Hutter from Wikipedia]
* [http://www.hutter1.net/ HomePage of Marcus Hutter]
=References=
<references />
'''[[People|Up one level]]'''
[[FILE:marcushutter.jpg|border|right|thumb|link=http://www.hutter1.net/index.htm|Marcus Hutter <ref>[http://www.hutter1.net/index.htm HomePage of Marcus Hutter]</ref> ]]
''Marcus Hutter''',<br/>
a German physicist and computer scientist, professor in the ''Research School of Computer Science'' at [[Australian National University]]. Before, he researched at [https://en.wikipedia.org/wiki/IDSIA IDSIA], [https://en.wikipedia.org/wiki/Lugano Lugano], [https://en.wikipedia.org/wiki/Switzerland Switzerland] in [[Jürgen Schmidhuber|Jürgen Schmidhuber's]] group. Marcus Hutter defended his PhD and BSc in physics from the [https://en.wikipedia.org/wiki/Ludwig_Maximilian_University_of_Munich Ludwig Maximilian University of Munich] and a [https://en.wikipedia.org/wiki/Habilitation Habilitation], MSc, and BSc in computer science from [[Technical University of Munich]]. He is author of the AI-book ''Universal Artificial Intelligence'' <ref>[[Marcus Hutter]] ('''2005'''). ''[http://www.hutter1.net/ai/uaibook.htm Universal Artificial Intelligence]''. Sequential Decisions based on Algorithmic Probability, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]</ref> , a novel algorithmic information theory <ref>[http://www.scholarpedia.org/article/Algorithmic_information_theory Algorithmic information theory - Scholarpedia]</ref> perspective, also introducing the universal algorithmic agent called '''AIXI'''.
=AIXI=
<span id="AIXI"></span>Quote from ''The AIXI Model in One Line'' <ref>[http://www.hutter1.net/ai/uaibook.htm#oneline The AIXI Model in One Line]</ref>
It is actually possible to write down the AIXI model explicitly in one line, although one should not expect to be able to grasp the full meaning and power from this compact representation.
AIXI is an agent that interacts with an environment in cycles k=1,2,...,m. In cycle k, AIXI takes action ak (e.g. a limb movement) based on past perceptions o1 r1...ok-1 rk-1 as defined below. Thereafter, the environment provides a (regular) observation ok (e.g. a camera image) to AIXI and a real-valued reward rk. The reward can be very scarce, e.g. just +1 (-1) for winning (losing) a chess game, and 0 at all other times. Then the next cycle k+1 starts. Given the above, AIXI is defined by:
[[FILE:aixi1line.gif|none|border|text-bottom|link=http://www.hutter1.net/ai/uaibook.htm#oneline]]
The expression shows that AIXI tries to maximize its total future reward rk+...+rm. If the environment is modeled by a deterministic program q, then the future perceptions ...okrk...omrm = U(q,a1..am) can be computed, where U is a universal (monotone Turing) machine executing q given a1..am. Since q is unknown, AIXI has to maximize its expected reward, i.e. average rk+...+rm over all possible perceptions created by all possible environments q. The simpler an environment, the higher is its a-priori contribution 2-l(q), where simplicity is measured by the length l of program q. Since noisy environments are just mixtures of deterministic environments, they are automatically included. The sums in the formula constitute the averaging process. Averaging and maximization have to be performed in chronological order, hence the interleaving of max and Σ (similarly to minimax for games).
=Selected Publications=
<ref>[http://www.hutter1.net/official/publ.htm Publications of Marcus Hutter]</ref> <ref>[http://www.informatik.uni-trier.de/~ley/pers/hd/h/Hutter:Marcus.html dblp: Marcus Hutter:]</ref>
==2005 ...==
* [[Marcus Hutter]] ('''2005'''). ''[http://www.hutter1.net/ai/uaibook.htm Universal Artificial Intelligence]''. Sequential Decisions based on Algorithmic Probability, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Marcus Hutter]] ('''2007'''). ''[http://www.hutter1.net/ai/aixigentle.htm Universal Algorithmic Intelligence: A mathematical top->down approach]''. Technical Report IDSIA-01-03 In Artificial General Intelligence, [http://www.hutter1.net/ai/aixigentle.pdf pdf]
* [[Joel Veness]], [[Kee Siong Ng]], [[Marcus Hutter]], [[David Silver]] ('''2009'''). ''A Monte Carlo AIXI Approximation'', [http://jveness.info/publications/arXive2009%20-%20a%20monte%20carlo%20aixi%20approximation.pdf pdf]
==2010 ...==
* [[Joel Veness]], [[Kee Siong Ng]], [[Marcus Hutter]], [[David Silver]] ('''2010'''). ''Reinforcement Learning via AIXI Approximation''. [[Conferences#AAAI-2010|AAAI-2010]], [http://jveness.info/publications/veness_rl_via_aixi_approx.pdf pdf]
* [[Tor Lattimore]], [[Marcus Hutter]], [http://www.informatik.uni-trier.de/~ley/pers/hd/g/Gavane:Vaibhav.html Vaibhav Gavane] ('''2011'''). ''Universal Prediction of Selected Bits''. [http://www.informatik.uni-trier.de/~ley/db/conf/alt/alt2011.html Algorithmic Learning Theory], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science] 6925, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Tor Lattimore]], [[Marcus Hutter]] ('''2011'''). ''Asymptotically Optimal Agents''. [http://www.informatik.uni-trier.de/~ley/db/conf/alt/alt2011.html Algorithmic Learning Theory], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science] 6925, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Tor Lattimore]], [[Marcus Hutter]] ('''2011'''). ''Time Consistent Discounting''. [http://www.informatik.uni-trier.de/~ley/db/conf/alt/alt2011.html Algorithmic Learning Theory], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science] 6925, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
* [[Tor Lattimore]], [[Marcus Hutter]] ('''2011'''). ''No Free Lunch versus Occam's Razor in Supervised Learning''. [https://en.wikipedia.org/wiki/Ray_Solomonoff Solomonoff] Memorial, [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science] 7070, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer], [https://arxiv.org/abs/1111.3846 arXiv:1111.3846]
* [[Tor Lattimore]], [[Marcus Hutter]] ('''2012'''). ''PAC Bounds for Discounted MDPs''. [http://www.informatik.uni-trier.de/~ley/db/conf/alt/alt2012.htm Algorithmic Learning Theory], [https://arxiv.org/abs/1202.3890 arXiv:1202.3890] <ref>[https://en.wikipedia.org/wiki/Markov_decision_process Markov decision process from Wikipedia]</ref>
* [[Peter Auer]], [[Marcus Hutter]], [[Laurent Orseau]] ('''2013'''). ''[http://drops.dagstuhl.de/opus/volltexte/2013/4340/ Reinforcement Learning]''. [http://dblp.uni-trier.de/db/journals/dagstuhl-reports/dagstuhl-reports3.html#AuerHO13 Dagstuhl Reports, Vol. 3, No. 8], DOI: [http://drops.dagstuhl.de/opus/volltexte/2013/4340/ 10.4230/DagRep.3.8.1], URN: [http://drops.dagstuhl.de/opus/volltexte/2013/4340/ urn:nbn:de:0030-drops-43409]
* [[Tor Lattimore]], [[Marcus Hutter]] ('''2014'''). ''[https://link.springer.com/chapter/10.1007/978-3-319-11662-4_13 Bayesian Reinforcement Learning with Exploration]''. [http://dblp.uni-trier.de/db/conf/alt/alt2014.html Algorithmic Learning Theory], [https://en.wikipedia.org/wiki/Lecture_Notes_in_Computer_Science Lecture Notes in Computer Science] 8776, [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
==2015 ...==
* [[Tom Everitt]], [[Marcus Hutter]] ('''2015'''). ''Analytical Results on the BFS vs. DFS Algorithm Selection Problem. Part I: Tree Search''. Australasian Conference on Artificial Intelligence, [https://pdfs.semanticscholar.org/1b4b/c878b2d068214e39b258ee250e5b8889e84c.pdf pdf]
* [[Tom Everitt]], [[Marcus Hutter]] ('''2015'''). ''Analytical Results on the BFS vs. DFS Algorithm Selection Problem: Part II: Graph Search''. Australasian Conference on Artificial Intelligence
* [[Marcus Hutter]] ('''2017'''). ''Universal Learning Theory''. [https://link.springer.com/referencework/10.1007%2F978-1-4899-7687-1 Encyclopedia of Machine Learning and Data Mining 2017], [https://en.wikipedia.org/wiki/Springer_Science%2BBusiness_Media Springer]
=External Links=
* [https://en.wikipedia.org/wiki/Marcus_Hutter Marcus Hutter from Wikipedia]
* [http://www.hutter1.net/ HomePage of Marcus Hutter]
=References=
<references />
'''[[People|Up one level]]'''
