In this roundtable discussion, we hear from Deep Sentinel’s talented team of hardware, mobile and back-end engineers and product managers in Taipei, Taiwan.
Why did you decide to work with Deep Sentinel?
Morgan: From my work with a local startup accelerator AppWorks Ventures, I know firsthand that it’s not easy building a startup in Taiwan. While there are a lot of IPOs (international procurement offices) here, and a few companies like Dell, Apple and HP have design and manufacturing offices here, the financial climate here is not conducive for startups. Selly (Deep Sentinel CEO David Selinger) helps to bridge and solidify our value prop to Silicon Valley investors. His legacy and the reputation of our company’s investors are tremendously appealing.
Front-end engineer—and first Taiwan employee—heads to work at Deep Sentinel’s Taipei office.
Starks: I had previously been a freelancer, and had my own startup—for which I had IP and even a US patent—but I eventually spent all my money. I learned about this job (hardware and software engineer) through Winston’s blog, and knew I had to jump on the opportunity. AI is the future.
JJ: I had been working in FinTech before Deep Sentinel, and agree with Starks. AI is the future, and this product is revolutionary.
CH: There are so many things we need to solve, so there’s an opportunity to learn a lot and do a lot of different things. Also, Winston and Selly are very open-minded so you feel free to share any ideas, which is not always the case in traditional Taiwanese companies.
What are the biggest challenges working with a startup based in California?
Martin: I’m one week in the job, so this is new to me! It’s my first startup. My biggest challenge is that with my manager Winston being in the US, I need to find a better way to communicate with him. It’s interesting and challenging.
Morgan: We are testing out software methodologies to develop hardware. Typically in hardware, everything is very planned out from concept to production. In our standups every morning, we design what we’re doing, so we’re learning as we go along.
In my role as a Technical Product Manager, I manage all outsourced vendors, so consistent communication is critical, since I am the central point of contact between them and our hardware team in California.
How do you ensure collaboration across time zones?
CH: We have a daily morning standup. In the afternoon, each of us on the software team records a video to share the progress we’ve made and pass this update to the US team, so they can pick up where they need to. We also use Zoom conferencing a lot.
Morgan and Martin enjoy Taipei’s cityscape from their work space.
How would you describe the team culture here in Taiwan? Is it same or different from the one in Pleasanton?
JJ: We have lunch together every day, since we’re a small team.
Starks: We’ve just moved into this new office, so there’s a lot of space for all of us, but we still have a lottery system for picking seats. There are some coveted spots by the window, which were won through this system.
Ching-Wa’s love of astronomy led to her involvement in creating this polar map of structures in the Universe, garnering her team a “Year’s Best Science Images” accolade from Discover magazine.
You have a PhD in Astrophysics and did research at Johns Hopkins University. What led you on this path?
I’ve liked astronomy since I was 13. I remember going to a local space museum for a primary school trip, and feeling an intense connection to what I saw. I was fascinated by the mysteries of the Universe, and since I always loved math, I decided to do my undergraduate and master’s degrees in physics, and eventually go for a PhD in astrophysics. Going to Johns Hopkins University to continue my research, and working under the father of astronomy databases— Professor Alex Szalay—was my dream come true.
How did you apply your PhD in real life?
Astronomy has what you would call a Big Data challenge. The Sloan Digital Sky Survey (SDSS) helps to make sense of it all: It has imaged many galaxies and provides the most detailed three-dimensional maps of the Universe ever made. I joined the SDSS after I completed my PhD, and had the chance to collaborate with mathematicians, statisticians and computer scientists on the galaxy classification project. Our idea was, if we could tell exactly how similar one image is to another image, we can better classify galaxies. Because a galaxy image is multi-dimensional (many pixels and colors), the problem turns out to be very challenging, and is one that requires machine learning algorithms and computational power.
How does this tie in to the work you do at Deep Sentinel?
At Deep Sentinel, we use AI-powered computer vision and deep neural network technology to deliver intelligent home security. Computer vision helps our cameras to identify what they see. The way we structure and solve the problem relies a lot on classification. My experience in analyzing galaxy images comes in handy.
The Sloan Digital Sky Survey ushered in astronomy’s big data age with its three-dimensional maps of the Universe. For her galaxy work at SDSS, Ching-Wa also had an asteroid named after her in 2013elligent home security. Computer vision helps our cameras to identify what they see. The way we structure and solve the problem relies a lot on classification. My experience in analyzing galaxy images comes in handy.
Previously, I built a system to classify images, making use of a computer. But in my head, I built the mathematical intuition for the classification problem. I thought about point distributions and how to cluster them in high-dimensional pixel space. This kind of thinking helps me to delve naturally into deep learning, a technology that allows us to cluster data by their classes, or in the case of Deep Sentinel’s technology, to answer the question, is this an image of a “Person” or a “Car”?
How are you training the machine learning system at Deep Sentinel?
We started by picking a deep learning model architecture. At Deep Sentinel, we use state-of-the-art models that provide the highest classification accuracies from recent ImageNet competitions. The second stage of training is to provide data to train our algorithms. In our case, that’s a set of images and their labels (for example, “Person” or “Car”), collected from the initial users of our cameras. Finally, we evaluate a model by testing it on images that have not been seen by the model before. Our goal is to drive down the false negative rate (the inability to identify an intruder when there is one) to as close to zero as possible, and to identify exactly what’s in the image. Then, once the Deep Sentinel system is used in the owner’s property, we’ll further improve the model by training it with additional images which are relevant only to that particular user.
A collage of the galaxies that Ching-Wa Yip has helped classify during her time at SDSS
What does it take to be a good data scientist? Does your background in Astrophysics aid your work in data science?
A good data scientist is a bit like a Renaissance man or woman who possesses a mix of backgrounds. S/he needs to be a competent programmer, know about data structures and algorithms, and have a solid understanding of math, statistics and probabilities. But being an expert in all of these does not guarantee success. The ability to think outside the box, and incorporate these subject matters in an exploratory manner, are also important. Especially at a startup (compared to a big company or academia, where you might work on a smaller part of a problem), you have to be a self-starter. When you can’t Google it, you need to step into the shoes of any number of experts in order to solve the problem.
Ching-Wa attends the Institute for Data Intensive Engineering and Science Symposium in JHU Bloomberg Center for Physics and Astronomy, 2013.
To understand the universe was no different. Astronomy is transitioning from a data thirst to a data deluge discipline. Domain astronomers have to come up with new ways to extract information from the data. They have to learn new technologies that were simply unnecessary in the old curriculum. Back then, I trained machines to classify complex galaxies. Today, I’ve brought that experience to teaching machines to recognize images of people, cars, and animals, in order to make home security more effective. My background as a modern astrophysicist has shaped me into becoming a better data scientist, and I’m excited about the work ahead!
Tell me briefly about your startup background and experience.
I was a startup engineer and cofounder of a software startup in Taiwan focused on providing consumer insurance information. While I had raised seed funding for this venture, the funding environment there at the time was quite barren. To keep bootstrapping my startup, I took on consulting for Fliptop in 2011–a Silicon Valley startup tackling online recruiting. They asked me to work for them in San Francisco, and this is how I began my journey in Silicon Valley and eventually made it my home. I ended up being Fliptop’s first engineering hire in SF besides the VP of Engineering. The rest of the engineering team was in Taipei, since one of the founders was a Taiwanese immigrant to the US, and had connections there. I held a wide range of responsibilities–from customer success to platform engineering to debugging. Because I worked closely with the Taiwan team, 14-hour days were the norm. I’d work the day in California, do a hand-off to Taiwan, and continue to work a second day. They were eventually acquired by LinkedIn in 2015.
How is startup life in Silicon Valley different from Taiwan?
“Reconstruction After Knocking it All Down: A Taiwanese Engineer’s Discovery of How Work ‘Works’ in Silicon Valley” by Winston Chen, published March 2015 in Taiwan
Fliptop was my first US-based company experience. I was so transfixed by the cultural gaps between working in Taiwan and San Francisco, that I decided to blog about this online. In particular, I was struck by the notable differences in organizational structure (hierarchical vs. flat), personal communication style (constrained vs. expressive) and management style (metrics-based vs. experience-based). Public interest in my blog grew to the point where I had 5,000 followers in 12 months. An editor approached me and asked me to write a book. This got published. The net effect of all this was that I suddenly gained recognition and publicity in Taiwanese tech communities. I didn’t know it at the time, but this stroke of luck would be key to acquiring talent and establishing work-related connections for future endeavors.
How has your work at startups helped to hone your AI expertise?
After working at Fliptop–which used machine learning to pick out the most likely deals among sales leads–I consciously decided to pursue startups that would push me to expand my AI expertise.
In 2015, I was working at Quid, which uses machine learning to ingest data from up to 3,000 articles to find patterns and commonalities, then communicates this through an interactive UI for businesses. There, I worked on infrastructure, formulating and implementing the service operation monitoring system. At Banjo, which uses AI and machine learning to filter ongoing events from social media live streams, I focused on platform engineering, designing and maintaining platform data pipeline web services.
The breadth of experience I gained at these startups contribute to what I’m working on today. Startups have always been my ideal work environment, because you get to build your experience and know-how, and you never get comfortable and lazy doing the same thing. As a co-founder, I think it’s imperative to know something about everything, and to be able to learn everything within a short period of time.
How did you meet Deep Sentinel founder David Selinger (Selly) and when did you realize that Deep Sentinel was a good fit?
Deep Sentinel Founders David Selinger and Winston Chen, at the company’s first movie night…with just the two of them
When I realized that I was in danger of growing bored at Banjo, I knew it was time for my next adventure. Some founders had approached me on AngelList, but I had also reached out to early start-up teams. I saw Selly’s post about starting a LinkedIn group on Deep Learning and reached out to him. When he described the idea behind Deep Sentinel to me, I grew very interested, since I had toyed around with home security at one point doing something similar with my old iPhones. When I spoke with Selly about this project, he thought it was pretty cool. He invited me over to do some code jamming. We worked on one project for several hours, and afterwards, decided that we were a pretty good fit. That was over a year ago, in August 2016.
What’s your role at Deep Sentinel today?
As the co-founder at Deep Sentinel, I like to have my hands in a little bit of everything. I lead our software engineering team across the bay area and Taiwan, as we build up our mobile, web, and embedded products. A key part of my team-building efforts include fostering and maintaining a sense of shared culture.
Communication is paramount. Our team in California moves quickly, and can sometimes take productivity for granted. One of our Taiwanese engineers was hired from California, and the first time he sat, in-person with the rest of the team in California, our productivity more than doubled. We closed all our open tickets. I realized then that we had to double-down on inter-office communication. So now, I work until midnight to communicate all tickets and answer questions. Then, when California is offline, our Taiwan team has quiet time to focus on what they’re doing.
Open communication lines means having a 24-hour live stream between the Taipei and Pleasanton offices. Naturally, this leads to some light-hearted moments in the day, too.
I’m also working on encouraging the team to speak out, and be willing to have confrontations or arguments, which requires some undoing on the cultural side.
Our CMO has also held regular meetings with our Taiwan team on identity, design and naming, to keep them in the loop. On subjective topics such as naming, they’ve been eager and comfortable voicing their opinions. We’re optimistic about building a shared culture from the get-go, and have had the chance to reinforce this with frequent trips to Taiwan by our CEO, our hardware engineering lead, and myself.
After nine years at the helm of RichRelevance–an Artificial Intelligence (AI) company focused on personalizing the online shopping experience–I was ready to return to my roots building technology and focusing on the next generation of AI and robotics. I hadn’t settled on an exact field, but in Lux Capital I had found the perfect partner with whom to explore opportunities and ideas. Lux invests in emerging science and tech on the cutting edge, oftentimes even before a business case is proven. As a VC having mostly PhDs as partners, they can walk the walk. While meeting and exchanging ideas with the folks at Lux, I happened on deep neural networks–known many decades ago as multi-layer perceptron networks–which led to my first “aha” moment in the founding of Deep Sentinel.
Aha #1: Deep neural network technology has finally come of age after decades of false alarms…and it’s the single largest technology leap forward I’ll see in my lifetime.
When I realized that the buzz out of Google on “deep learning” was real, I was blown away; most “AI” hype is marketing hullabaloo. Deep Learning is anything but that. Originally conceptualized in the ‘90s, we didn’t have the underlying technology to make deep neural networks work back then. But today, with a series of leaps forward in technology, deep-learning powered machines can learn abstractions that until now have been the exclusive domain of human beings. You already encounter this when Facebook automatically tags your friends, but deep neural networks can do so much more–even things like restoringcolor in black-and-white photos, or synthesizing audio with lip motions in videos. A combination of underlying technology and statistical method improvements led me to the conclusion that deep learning will be the single largest AI leap forward I expect to witness in my lifetime.
Aha #2: In today’s home security market, “advanced technology” is still a euphemism for wireless sensors and megapixels.
The next “aha” moment occurred not long after a burglary took place in my neighborhood. As a result of this crime, I researched home security systems, and I was frankly appalled by what I found. “Advanced technology” in the legacy world of home security was basically a euphemism for wireless sensors and megapixels!
Just picture the ADT technician in my home, excitedly demonstrating their “advanced technology” by highlighting an LCD screen, keypad and wireless sensors. This “Back to the Future” moment felt like I had been dropped into an ‘80s TV infomercial.
And that’s when the final “aha” came.
Aha #3: Deep neural networks can recast and solve today’s legacy home security problem.
My research into deep neural networks and the underwhelming state of the so-called “home security” market catalyzed my final aha moment. Ninety-nine percent of crimes occur after the intruder has crossed the property perimeter. If we could use AI to cost-effectively detect and evaluate threats at this crucial juncture, we could tilt the advantage from criminals to homeowners–creating a new class of home security solutions. Using real-time AI, imagine activating warnings in the form of sound or light to deter criminal activity within seconds, instead of just recording it to review tomorrow over a frustrated shout of “DID YOU SEE THAT!” or a “STOP, no, I wish I could go back in time!”
Much like when I co-founded Redfin, I see the opportunity for technology to transform the fundamental expectations of a market. At Redfin, interactive mapping, UX and transparency of data (i.e., Big Data) revolutionized the real estate industry by starting with the consumer. Similarly, at Deep Sentinel I see deep neural networks and computer vision transforming every consumer’s expectation of a true sense of home security.
Many aha moments later, Deep Sentinel is one year in and up and running. After testing many prototypes with off-the-shelf cameras and getting a series of AI clusters going, we remain fueled by that same founding vision of using AI to keep the bad guys away from people’s homes. We’re blessed with the support of great investors like Jeff Bezos, Pierre Omidyar, Lux Capital and Shasta, as we continue to build our team of deep learning computer vision and hardware experts. We hope you’ll follow along, and share in this journey with us!