Sep 9 2019 "How I landed a role as a Machine Learning Engineer in Japan"
One of the booming sectors in Japan is the tech industry especially with technologies involving Artificial Intelligence and Machine Learning. Software Engineers, especially those with doctorate degrees, are highly sought after and companies are very competitive in getting the right candidate.
We interviewed Dr. Aaron Bell, a former Active Connector candidate, regarding his career journey as a fresh Ph.D. graduate majoring in Astronomy. He shared some tips and insights based on his experience as a foreign talent job hunting in Japan
We divided this interesting discussion into 2 parts. The first part is all about Aaron talking about how he built his skills and knowledge that was helpful in transitioning to a Machine Learning Engineer role. Here he shared in detail the main differences between handling data within the Academia as a student and as an official employed Machine Learning Engineer in the Workplace. This includes his job hunting experience and the motivation behind him moving to Japan and testing the waters.
Building up skills and knowledge to transition to AI/Machine Learning
Please tell us what brought you to Japan?
Dr. Aaron Bell: “Towards the end of undergrad, I knew I wanted to try living abroad. I’d had my first overseas trip to Brazil for an astrobiology conference, and loved it. I’d been taking Japanese as my foreign language requirement and loved that too. I know if I was going to try grad school somewhere far away, and different, it may as well be really far away. At the same time I was a coward, not wanting to go somewhere that would be too much of a shock. Japan seemed like a good fit.”
How did your journey here start?
Dr. Aaron Bell: “I came here first as part of the University of Tokyo Research Internship Program (UTRIP)– basically a 6-week preview of the life of a Todai grad student. Actually, I got rejected the first time I applied. I waited a year, applied again, and things worked out the second time.”
What was your motivation to move here?
Dr. Aaron Bell: “The polite conversation answer is Grad school. I applied to do a masters degree in the same lab in which I worked during UTRIP. This was a much heavier decision than just spending a summer here, but I just felt that I wasn’t the kind of person who could just do an internship in a far away, interesting place, and then go back to the country I grew up in and be satisfied. If I was going to do the international thing, I wanted to do it long enough, and early enough in life, that it’d shape how I grew up.
Underlying all that though, was a more personal motivation. As a gay individual growing up in ultra-conservative Southern Kentucky, I was always wishing I could get away somewhere I felt I was free to be myself. Sure, Japan is pretty conservative in its own ways — many of these I didn’t realize until the last few years — but in many ways I feel safer, and have more license to be myself here than back home. (Surely being a white male still helps with that, whether people know my orientation or not…)”
Your background in Astronomy is very interesting, How did you decide to go from Astronomy to Machine Learning?
Dr. Aaron Bell: “It was pretty natural really. First let me confess, even as an astronomer I never used a telescope during grad school. I worked with data archives from space telescopes, mainly one launched by JAXA. In fact, by tune my masters degree started, the space telescope that had taken my main data had long since finished its mission and has been deactivated. The statistical and computing tools I used to analyze and present that data, are not so different from tools used in data science and machine learning projects. I started out trying to apply machine learning to astronomical data in my PhD course. That eventually led my to widen my imagination, and think about doing similar work but not limiting myself to astronomy (It might sound weird but the universe is bigger than astronomy).”
Was it difficult to transition from your subject of expertise (Astronomy) to Machine Learning and what you are doing at your workplace?
Dr. Aaron Bell: “In many ways it was really easy– not that I’ve finished the transition yet! In both cases I have to solve problems with statistics, data management, and visualization. Both require me to explain what I’m doing both to experts and to people of completely different backgrounds. In principle we follow the scientific method in both cases. The major difference is the pace, the amount of feedback, and customer’s goals. In astronomy, you sometimes have to make a paper referee happy, but you’re not writing a paper to make the referee money. The referee can’t force you to write on a different topic. They don’t have regular face-to-face meetings at your office to make sure you’re not wasting their money. But there is something surprisingly similar– managing expectations:
When you write a scientific paper, you have to define your scope. This is what I’m doing. This is what I’m not doing. I could have done A, Yes A would be interesting, but other people are working on A, therefore I’m working on B. If you are interested in A, please talk to someone working on A. In fact a lot of this happens before you even write the paper, by way of choosing the type of conference you visit and present at, which journal you submit to, which co-authors you work with. This is sort of an academic way of managing expectations of your work, so that you referees judge your work based on the paper you wrote, not the one you didn’t write.
Not only the referees, but your adviser, and yourself. You have to constantly negotiate what’s expected from a particular experiment, especially in the case that it fails or you get a negative result. You have to always consider what’s the minimal potential output of your research. (That never stops me from hoping for an exciting outcome…)
In the ML business, those same principles apply. There’s just perhaps a few added layers. At the top, the company itself has to manage what its clients expect. We have to make sure they know which parts of what we do are predictable— “if you have as X amount of time, within some error, we can deliver Y.” And which parts are R&D “Even if you give us 1 year, we can’t promise this experiment will succeed…” or in some cases “What you’re asking is really cool, but impossible…. we can’t take your money for this”. For the engineer, we’re mostly managing the project manager’s expectations (and our own). The main point of course is time — “I think this will take me 3 days” or “If you want this by tomorrow, I’m going to need 3 hours of Engineer C’s time”. But also expertise. Whenever a manager asks me to do something new, I make sure to give a disclaimer: “I’ve never done this before. It might take me longer than expected. It might fail, so let’s go ahead and make a Plan B”
I also have to manage my own expectations– it’s good keep making checkpoints for yourself. What do you expect to have achieved by a certain time frame. Write it down on your calendar. When the day comes compare what you wrote to reality. It doesn’t have to be accurate, but the point is that, like a Machine Learning algorithm, you start to train yourself as to which goals are unrealistic or vague, which ones are pessimistic, or which ones are just imaginative enough to keep you motivated.”
How did you handle this transition?
Dr. Aaron Bell: “There’s a lot more emphasis now on code readability, efficiency, debugability, explainability, and intermediate goals. Those things are of course really important in academic astronomy world too– but the simple truth is it’s easier to get away with lazier code when you’re working alone on a long-term project.
Otherwise, the real “work” of my day is very similar to my PhD student days; Constantly learning new coding techniques, tweaking plots, searching for improved ways of solving problems, challenging myself.
In some sense it’s easier than before, because there’s always someone around to ask engineering questions. As an astro grad student, there were a lot of experts around, but not many people skilled in the kind of large scale data analysis I was doing, which made it really hard to get advice and learn new techniques.
The major difference in style is just the timescale. I try to do my work incrementally, trying to have some minimal but measurable output for the next progress report. In academia I could embark on more long term experiments, take more risks with my time. That’s not to say I can’t experiment at work, but just that I think more carefully about making sure I have some minimal contribution before I diverge too far from the task at hand. I also have to stop regularly to think about if the direction I’m going with a piece of code is going to do something meaningful for the customer, or is it just something I’m coding because it’s personally interesting. It’s always a balance.”
What were the skills that were transferable that was vital to be a Machine Learning engineer?
Dr. Aaron Bell: “Data visualization is probably the biggest one. In technical terms plotting in python, matplotlib.”
When it comes to starting your first official job, it can be overwhelming especially when you do it in a country with a totally different culture like Japan. But you might be surprised with the countless opportunities stored for you especially if you have attained a higher education and have a technical background.
Ready to take the next step? Submit CV and gain access to hundreds of handpicked jobs and opportunities across Japan today.