Ben Duffy's adventures with Computers and Music

Click to unroll/roll all sections

Links

Short Bio

Roboticist, AI Researcher and Computer Engineer hybrid obsessed with using whatever is necessary to create truly intelligent machines for the benefit of humanity.

Co-founder at thing and MTank

My journey through technology:
  • Hobby Game Development (2014-2017)
  • Web Development (2015+): Every job I have had required it
  • Computer Vision (2015+)
  • Deep Learning (2014+)
  • Data Science and Engineering, NLP (2017-2019)
  • Reinforcement Learning (2017-2019)
  • Full-stack Robotics (2019+): hardware (actuators, sensors, mechatronics and rapid prototyping) and software (perception, control, SLAM, navigation, manipulation, integration and testing). I love this because everything I have learned previously coalesces and accumulates to be useful to some extent for this field.
  • Radar and Digital Signal Processing (2021+): Trying to sense breathing through walls

Formerly worked at Accenture as a Data Scientist working on projects which included:
Expense Anomaly Detection, Marketing Analytics, Pricing Optimisation, Recommender Systems, User modelling with Knowledge Base Graph Embeddings and resume to job matching.

Publications

Blogs

Reports/papers

Short timeline

  • 2020: Moved to Berlin with the two other founders of MTank to go full-time into creating thing (www.thing.house).
  • 2019: Left Accenture in May 2019 to begin properly learning and working on robotics projects.
  • 2017: Started as Data Scientist in Accenture at the Dock Innovation Centre in March.
  • 2016: Finished 4th year Computer Engineering at TCD. Founded MTank. Did a 3 month internship at Intuition as a Full Stack web developer. After that, 6 month "hack sabbatical" to upskill for a job in Data Science.
  • 2015: Got my first job (and best possible first job ever) as a full stack web developer intern working at Pointy.
  • 2014: Finished 2nd year General Engineering and specialised into Computer Engineering for the last 2 years.
  • 2012 and before: Birth -> Kindergarten -> Primary School -> Classical Guitar (age 7) -> Secondary school -> became good at math and music -> had a crisis dichotomy about which career path to choose out of both but "accidentally" chose engineering -> Started 1st Year General Engineering

Full timeline

  • 2020: Moved to Berlin with the two other founders of MTank to go full-time into creating thing (www.thing.house). Involved with every part of what is needed from full-stack robotics i.e. hardware (actuators, sensors, mechatronics, rapid prototyping and manufacturing) and software (perception, control, SLAM, navigation, manipulation, integration and testing). Will release videos over the months.
  • 2019: Released: Website: ai-distillery.io/. Gave a talk on "An Introduction to Reinforcement Learning" for the ODSC meetup. Organised the first MTank meetup in Stuttgart on "AI Distillery and the Journey from Cups to Consciousness". Went to probablistic.ai and RSS2019. Left my Data Science job in Accenture after 2 years to go "all in" into Robotics with MTank (our 2nd vision) and a stealth startup I'm founding. Exciting stuff! Spoke at an RPA conference and Ancestry.com meetup in September and Predict - Europe’s Leading Data Conference.
  • 2018: Released Multi-Modal Methods. Gave my first 4 talks (1, 2, 3, and as one of the winners of a Computer Vision hackathon at 4). Was on my first podcast. Finished my Classical Guitar Diploma after 17 years of training. 3 summer schools (ISSonDL, DLRL2018 (summarised into 30 tweets by me here and slides here), DeepBayes) and my first conference (HLAI). Created a 4 hour Reinforcement Learning course at: RL course. Began working on two big projects within the MTank:
    • From Cups to Consciousness (c2c). Training embodied agents for multi-task learning through natural language instructions in 3D simulated houses and eventually the real world. On GitHub here: cups-rl using ai2thor.
    • AI Distillery. Automatically modelling and distilling knowledge within AI. In other words, summarising the research firehose coming from related fields within ML/DL/RL/AI/DS/CS/Stats using embeddings (doc2vec, word2vec), information retrieval and visualisation techniques. Check out some of our work at: ai-distillery.io
  • 2017: Accepted offer from Accenture (at the Dock) for a Data Scientist role in March. Skills/things learned:
    Machine Learning, Deep Learning, AI, Analytics, Visualisation, Data engineering, Research, API and full-stack Web Development, Business skills and a lot of domain knowledge for each specific project.
    Language/communication skills (speaking, listening, writing, reading, thinking, blog writing, general storytelling and presentation skills, UX/UI/Design Thinking skills and Agile/scrum).
    Released A Year in Computer Vision. Went to my first summer school at DeepLearn2017. First TV interview (see my YouTube or messy project gallery).
  • 2016: Finished 4th year Computer Engineering at TCD, learned SQL and did a module in Augmented Reality. Watched the cs231n videos. Finished my undergraduate thesis on "Video Understanding" for which a first-class honours was received. Convolutional Neural Networks were used for Object Detection, Scene Classification and I used a statistical method for Shot Transition Detection. Results were shown in a web interface for any YouTube video the user chooses.
    Founded MTank and increased the rate at which I could read and extract information from the abstracts of thousands of papers, while skimming a select few and properly reading an even smaller subset. Began learning how to play the tenor saxophone. Did a 3 month internship at Intuition as a Full Stack web developer. After that, I took 6 months off for a "hack sabbatical" of intense studying at home (coding, courses, meetups, hackathons, writing, researching) mainly for fun and to upskill for some kind of job in ML/AL/DL/DS.
  • 2015: Finished 3rd year Computer Engineering. Did more projects in Computer Graphics (C++ and OpenGL), game development (Java, Android and LibGDX) and Computer Vision with OpenCV. Check the messy project gallery for pictures and videos of most of these. Got my first job (and best possible first job ever) as a full stack web developer intern working at Pointy. I didn't know Python or JavaScript before starting and now both are my favourite languages. Created this website.
  • 2014: Finished 2nd year General Engineering and specialised into Computer Engineering for the last 2 years. Suddenly got much better by banging my head against a wall and started to absolutely love programming. Plenty of side projects, reading and learning in my spare time. Made a few games using DirectX and went deep into C++. Starting learning Java, Android and OpenGL.
  • 2013: Sulked for a while and regretted not choosing music which was my first passion. Finished first year General Engineering and started 2nd year. Chemistry, Physics, Mechanics, etc. Did two modules on programming. After the first I thought I was bad at it, but I tried harder and practiced and then started to like it.
  • 2012 and before: Birth -> Kindergarten -> Primary School -> Classical Guitar (age 7) -> Secondary school -> became good at math and music -> had a crisis dichotomy about which career path to choose out of both but "accidentally" chose engineering -> Started 1st Year General Engineering

Principles

  • My two favourite things to do are: Learning and Creating. In and out. Take and give. They also both naturally complement each other i.e. the more you create, the more you learn. The more you learn, the more you CAN create.
  • Lifelong and Continual Learning. Learning is the spice of life. Learning is how your brain grows. Learning can be one of the greatest providers of happiness. Learning keeps dopamine levels high and gives incredible satisfaction that can last a lifetime. Therefore, in my opinion Learning is the best drug.
    Transfer Learning i.e. it is easy to see how learning and practising ping pong might help your tennis skills but I'd push the concept as far as saying that practising ping pong will help your essay writing skills. Now I understand this is a push but I think learning anything has deep, indirect, hidden and countless connections with everything else you learn/process subconsciously and consciously but this might be hard to accept so I will throw the concept of Meta-Learning into the picture.
    Meta-Learning i.e. Learning how to learn. Once you learn how to learn well, you can learn anything. How do we get better at learning itself? By practicing learning constantly while simultaneously analysing and finding better ways to improve your own personal learning. Therefore, learning anything can help your ability to learn other unrelated things (Transfer Learning again and I think it makes my "ping pong -> essay writing" point at least somewhat true in this sense). It is sad that the state of education today often discourages learning more than it encourages it. #feelthelearn
  • Creating... So much to say about this but I'll leave it at: very little can give more meaning and fullfilment than the act of creating something new directly from your own imagination. Learning is great and all, but only your creations will live on after you die i.e. they are your offering to the world and they carry on your legacy.
  • I like to make everything as simple as possible by using logic, breaking it into sub-parts, reorganising, focusing and using a first-principles approach. This helps me understand things to the very inner core and throughout my life this has helped me and proven to be very effective to everything from music to programming to reading to writing to understanding very complicated ideas etc.
  • Consistency and Focus is key. Great good can be accomplished through focus. It's not only how hard you work, it's also how smart and focused you work. One man's hour can accomplish what takes another man ten. Also on a related note, the 10,000 hour rule can be very discouraging to many people, you can get subjectively "excellent" at things within 100 hours (or at least get a "Woooow man, you're really good at that!").
  • Two factors to personal success: Belief and Hard Work (which is Smart, Focused and Consistent as well). Self-help books complicate this by making this list 100 points long. Also, of course belief often leads to motivation, passion, laser focus, etc.
  • Two factors to business success: A good idea and good execution of said idea. Everyone and their grandmother has 20 great ideas, good execution is much harder to find.
  • More

Programming Languages

Main used languages in order of preference/frequency used

  • Python
  • JavaScript
  • C++
  • Java
  • C#
  • Matlab/Octave
  • SQL
  • Assembly (ARM and IA32)

Other "Programming Languages" used

  • HTML
  • LESS/CSS
  • JSON
  • XML (XPath, XSL)

Fields/Specialisations

Machine Learning/Data Science/AI

This is my main passion as this is currently how one can solve the world's toughest problems. I love to play around with data. And as I said above in my principles - All that there is, that a computer does/has, is a list of two things: Computation and Data. I want to master both.

Main fields/areas within AI

  • Robotics. This is my main focus now.
  • Deep Learning. I intensely follow around 2000+ papers a year in this and similar areas and I'm very up to date on the current state of the art of the field. I completed my Bachelor's thesis on using DL for "Video Understanding"
  • Computer Vision
  • Reinforcement Learning
  • Natural Language Processing
  • Generative Modelling and Unsupervised Learning

Other fields/areas that I'm interested in

  • Meta-Learning
  • Bayesian modelling and Graphical Models
  • Few-shot Learning, Semi-supervised Learning, Transfer Learning, Multi-task Learning, Continual/Lifelong/online learning
  • Hierarchical RL, Imitation Learning, Multi-agent RL, Inverse RL
  • Neural Program Induction/Synthesis
  • Domain Adaptation and Cross-modal/cross-domain Learning
  • Memory Augmented Learning
  • Reasoning/XAI/Common Sense/Physical Experiments/Graphs

General Algorithms

  • Linear Regression, Logistic Regression, SVMs, Random Forests/Decision Trees, K-means, K-nearest Neighbours, Matrix Factorisation, Naive Bayes, Dimensionality Reduction (PCA, t-SNE)

General Data Science skills

  • Probability/Statistics
  • Data Visualisation
  • Data Cleaning
  • Web Scraping
  • Research. Reading AI research papers and keeping up with all the latest developments in technology as a whole but with a focus on AI and all fields it is related to.

Deep Learning Frameworks

  • PyTorch
  • Keras
  • TensorFlow
  • Caffe
  • ConvNet JS

Other Frameworks/Libraries

  • NumPy
  • pandas
  • Scikit-Learn
  • matplotlib
  • seaborn
  • bokeh
  • D3.js
  • OpenCV
  • Jupyter/Ipython

Graphics, Visualisation and Game Programming

I have made games in Java, C++, Python and JavaScript using the below frameworks. I have used D3 for visualisation and generally just like to make pretty looking things.

Frameworks/Engines/Libraries

  • LibGDX
  • Phaser.js
  • DirectX
  • OpenGL + OpenGL ES
  • WebGL
  • Unity
  • pyglet

Web Development

Client Side Frameworks/Libraries

  • jQuery
  • Bootstrap
  • Templating with Handlebars, Jinja2
  • Angular JS (1 and 5)
  • Google Maps API
  • D3.js
  • Leaflet JS
  • WebGL
  • LESS

Server Side Frameworks/Libraries

  • NodeJS
  • Express
  • Google App Engine (using Python and webapp2)
  • Python Flask
  • Socket.io
  • Grunt
  • Azure PaaS
  • Consul

General Web skills

  • Security (SQL injection, Cross site scripting, TLS, etc)
  • HTTP + DNS
  • Microservices
  • Go Continuous Delivery
  • RabbitMQ
  • NoSQL (Cassandra, MongoDB, App Engine Datastore)
  • SQL (MySQL, T-SQL, Oracle SQL)
  • REST (documenting APIs, creating/debugging APIs, Postman, Fiddler, etc)

Other technical skills

Operating Systems

  • Windows
  • Linux
  • OSX

Other

  • Git
  • Android development
  • Agile
  • Design Patterns
  • Video editing, powerpoint, graphing/mindmap tools
  • Testing (Unit, Integration)
  • Microservices
  • Continous Delivery/integration
  • IDEs: PyCharm, Visual Studio, Sublime Text, Atom, Eclipse, Android Studio, WebStorm, IntelliJ, IDLE Python 3, Arduino, Keli uVision 4. Can sometimes, on a good day, exit Vim

Music

Instruments

  • Classical Guitar ()
  • Electric Guitar ()
  • Piano ()
  • Saxophone ()

Bands and projects

  • The Brickfield Park trio (Jazz/Funk mostly)
  • Panic at the Bustop (Busking acoustic guitar duo)
  • The Hacks (Alternative Rock)
  • The Scruffs (Alternative Rock/Rage Against the Machine covers)
  • SoundCloud

Other Interests

  • Generalisation
  • Specialisation (it doesn't have to be only one or the other)
  • Philosophy
  • Psychology
  • Engineering and Science in general
  • Technology applied to anything
  • Business/Economics/Finance
  • All five uses of language: Listening, Speaking, Reading, Writing and Thinking.
  • Neuroscience
  • Rockets and space
  • The Future
  • The Present
  • The Past
  • Listening to music
  • Travelling
  • Geography
  • Books
  • Films
  • Comedy
  • Cycling
  • Exercise
  • Almost everything else