What is Zero Shot Learning?

Zero-shot learning is a technique in machine learning where a model is trained to recognize objects or concepts it has never seen before. Conventional machine learning models require a large amount of labeled data to train, which makes it difficult to recognize new objects or concepts that were not part of the training set. In contrast, zero-shot learning models can recognize new objects or concepts by leveraging prior knowledge about related objects or concepts.

In zero-shot learning, the model is trained to recognize a set of known classes or concepts, but it is also trained to understand the relationships between these classes. The model is then able to recognize new classes that are related to the known classes, even if it has never seen them before. Zero-shot learning is particularly useful in situations where it is difficult or impractical to obtain labeled data for new classes.

How Does Zero Shot Learning Work?

Zero-shot learning works by leveraging a technique called attribute-based classification. In attribute-based classification, an object or concept is described by a set of attributes, such as color, shape, size, texture, etc. Each attribute is assigned a score or weight that represents its importance in describing the object or concept. The model is then trained to recognize objects or concepts based on their attributes.

To recognize a new object or concept, the model first identifies its attributes and their scores. It then compares these attributes and scores to the attributes and scores of the known classes. The model selects the class that has the most similar set of attributes and scores as the new object or concept.

Applications of Zero Shot Learning

Zero-shot learning has many potential applications in various fields. Some of the most promising applications include:

Image Recognition

Zero-shot learning can be used to recognize objects in images, even if they have never been seen before. This has applications in fields such as autonomous vehicles, where the vehicle needs to recognize new objects on the road.

Natural Language Processing

Zero-shot learning can be used to understand and generate natural language text. This has applications in fields such as chatbots and virtual assistants, where the system needs to understand and respond to new user requests.

Medical Diagnosis

Zero-shot learning can be used to diagnose medical conditions based on symptoms, even if the condition has never been seen before. This has applications in fields such as telemedicine, where doctors need to diagnose patients remotely.

Conclusion

Zero-shot learning is a promising technique in machine learning that allows models to recognize new objects or concepts that were not part of the training set. By leveraging prior knowledge and understanding the relationships between classes, zero-shot learning models can recognize new classes with high accuracy. Zero-shot learning has many potential applications in various fields, including image recognition, natural language processing, and medical diagnosis. As the field of machine learning continues to evolve, zero-shot learning is likely to become an increasingly important technique for solving complex problems.

This article written by a large-language model.

People are different, and while some may perform exceptionally well in one environment, they may struggle in another. Rather than viewing the act of letting go of employees as a failure, what if we looked at it as an opportunity for growth and development? By regularly changing teams and companies, we give employees the chance to try out new roles, explore different industries, and ultimately find their true calling.

Implementing a regular team/company change policy could also lead to a more diverse and dynamic workforce. By encouraging employees to work in different teams and departments, they can learn new skills and gain a better understanding of the company as a whole. This, in turn, can lead to increased collaboration, innovation, and ultimately, success.

According to a study by the Society for Human Resource Management (SHRM), the cost of a bad hire can be as much as five times their annual salary. Additionally, a Gallup study found that disengaged employees cost US companies up to $550 billion per year in lost productivity. These staggering numbers highlight the importance of ensuring that every employee is a good fit for their role and the company as a whole.

One real-world example of a company that embraces regular team and company changes is Amazon. The company has a policy called "pivot or perish," which encourages employees to move on to new roles and teams if they feel that they are no longer growing in their current position. This policy has been credited with helping Amazon maintain a culture of innovation and adaptability.

Another example is Zappos, which has a policy called "the offer." After completing their initial onboarding and training, new hires are offered a $2,000 bonus to quit if they feel that Zappos isn't the right fit for them. This policy ensures that every employee is fully committed to the company's values and culture.

By embracing regular team and company changes, we can also create a more inclusive and diverse workforce. For example, a study by McKinsey & Company found that companies in the top quartile for gender diversity are 15% more likely to have financial returns above their respective national industry medians. Similarly, companies in the top quartile for racial and ethnic diversity are 35% more likely to have financial returns above their respective national industry medians.

In conclusion, making letting go a regular part of business is not only necessary but also beneficial for companies and employees alike. By creating a culture that values growth, development, and diversity, we can build stronger, more adaptable, and more successful organizations.

Leave a comment

AI Writes is a newsletter entirely generated by ChatGPT — prompted with ideas by a human named Param Aggarwal (paramaggarwal.com)

If you work on a React Native based mobile app, you most likely are using a Mac to be able to test your app on iOS as well. This means that even though the SSD is very fast, it is either 128GB or 256GB. This can get easily filled up due to various caches that are made by various build systems.

This article will point you to the right places to help clear up the caches and rebuild only what is needed.

First, go to  > About this Mac > Storage and then choose “Manage…” on top right.

Now choose “Review Files” on bottom left and then select the File Browser tab. At first glance, you will notice that in the left sidebar, it will show that majority of the disk space is consumed by the “Other” category.

This is because all the cache directories are in the home folder and are hidden folders starting with a dot in the name. This causes them to not be easily searchable.

The fix to getting the system to list all hidden folder as well is to press CMD + SHIFT + . — this will cause all the dot folders to also show up. Now it will begin calculating the size of each directory and updating the list.

You can safely get rid of the following directories by right clicking and selecting Move to Bin option:

• ~/.npm — npm cache

• ~/.yarn and ~/Library/Caches/Yarn — Yarn cache

• ~/.gradle — Gradle cache

• ~/.m2 — Maven cache

• ~/.cocoapods and ~/Library/Caches/CocoaPods — CocoaPods cache

• ~/.nvm — Node Version Manager cache

• ~/.android — Android cache

• ~/Library/Caches/Homebrew/downloads — Homebrew cache

• ~/Library/Developer/Xcode — Xcode cache

• ~/Library/Developer/CoreSimulator — iOS Simulator cache

In addition to all of these, in your individual project repos, you can delete these folders of less used projects, they can be easily recreated if needed:

• node_modules — Just run npm install again when needed

• android/app/build — Just rebuild the Android app when needed

After deleting all the folders above, you should easily be able to reclaim around 100GB of free space. Make sure to run the following command in your primary project to rebuild the caches — this will take time the first time.

• npm install or yarn in your primary projects

• pod install in your project’s iOS directory

• .gradlew build in your project’s Android

With all of this done, you should have a lot of free space for many weeks before the caches begin to fill up again! Then you can come refer back to this guide to help free up disk space on your Mac laptop.

Negative thoughts compound. The more you think of yourself as worthless, stupid, or ugly, the more you condition yourself to interpret life that way. You get trapped in a thought loop.

Why do we expect so much from ourselves?

Often we feel that the most difficult part about improving is to figure out what to do. When we see someone successful we tell ourselves that we also want to be the same and then we try to copy the most superficial aspects of the person in order to attempt to replicate their success. This is why we set such unrealistic goals on ourselves. Also we fall into the trap of thinking that if we think it, it will be done.

Why do we then feel worthless when we fail to achieve these goals within a matter of few days?

Because it is easy find logic in our thoughts and ideas. As an example, it is easy to read an article that says that breathing slowly helps improve mindfulness and then we tell ourselves I am going to breathe slowly every day.

In that moment we feel we have made a sensible target and that now onwards we will change this aspect about ourselves. But actually we have defined nothing. Hence the next day it is easy to feel bad that we didn’t do anything about it. Actually we never defined what was to be done in the first place.

We only defined what we want, not how to get it.

Why do we then form a negative image of ourselves for not being able to accomplish things?

This I believe stems from an unreasonable expectation of plasticity from self. That is, that if I think I will do B instead of A every day then magically I will be able to stick to this thought process. But humans are creatures of habit and they will go back to doing A very quickly.

Hence once we do this deep thought exercise we realise that we should:

1. Read a lot

2. Not fall into the trap of setting arbitrary goals of “I’ll become this different persons from now”

3. Not self-disrespect self and speak of self in third person. Be kind to self.

4. Focus on moments rather than end-of-day retrospects. What should I do right now? If you don’t have answer then build a habit plan and refer it when in doubt.

With this foundational thought exercise, we should now read this again.

Negative thoughts compound. The more you think of yourself as worthless, stupid, or ugly, the more you condition yourself to interpret life that way. You get trapped in a thought loop.

On reading this the second time, we learn that we think of ourself as worthless, because we don’t see ourself as ourself. We see ourself as the person in front of us in that moment who we want to be like.

We feel stupid because we feel that converting a goal into actionable steps is trivial and anyone could do it.

We feel ugly because we are always looking at the best quality in the other person and grading ourself against that peak attribute of the other person.

You get trapped in a thought loop.

Now every time we look at someone, we are conditioned to think that there is a vast expanse between where I stand and where the other person stands and I will never be able to get there.

Why practice?

• Painting is a link from mind to brush.

• Writing is a link from mind to words.

• Driving is a link from mind to wheel.

• Cooking is link from mind to pan.

Thinking and doing becomes one.

In July 2005, Google acquired Android Inc. for at least $50 million. Not much was known about Android Inc. at the time, but many assumed that Google was planning to enter the mobile phone market with this move. At Google, the team led by Rubin developed a mobile device platform powered by the Linux kernel. Google marketed the platform to handset makers and carriers on the promise of providing a flexible, upgradable system. Google had lined up a series of hardware component and software partners and signaled to carriers that it was open to various degrees of cooperation on their part.

Speculation about Google’s intention to enter the mobile communications market continued to build through December 2006.[33] An earlier prototype codenamed “Sooner” had a closer resemblance to a BlackBerry phone, with no touchscreen, and a physical, QWERTY keyboard, but was later re-engineered to support a touchscreen, to compete with other announced devices such as the 2006 LG Prada and 2007 Apple iPhone.

At Google I/O 2014, the company revealed that there were over one billion active monthly Android users, up from 538 million in June 2013. How many of them have the Play Store installed? How many of them can Google ship a software update to directly? Android is a platform, it has far too many variants to list.

Google bought Android, the company in 2006 — started by Andy Rubin.

Google bought Android in 2006. Android started out by cloning the then popular Blackberry. because Reason being they wanted control on mobile platforms, so that their services were not side tracked.

They quickly re-align strategy to copy iPhone instead by 2008. By 2012, they have spent more than 10 billion buying Motorola to have a patent portfolio and develop Android, still giving it away for free in *hope* of future profits.

Today, Android is great. But the two companies with a majority of the smartphone market are Apple and Samsung.

Samsung day-by-day threatens a move to another platform, or fork Android (like Kindle) with it’s own default apps, and a store for media and apps.

Apple on the other hand is still the platform where Google makes most of its money with advertising from the default search, in-game advertising, and dedicated native apps for most of its services.

So how did Android help Google?

US citizenship is based on birth. So if companies are still picking immigrants to do the job, it is more a sign of the availability of skilled people.

US does not need to do manufacturing — they have moved on to doing high level design. Manufacturing is commoditized.

During this time of change, people who are on the gaining edge — the west and east coast of the US, which does fashion design, industrial design, software design — are all happy and see the prosperity that the US is gaining. They were fortunate to have education in the relevant areas that eventually made it big.

On the other hand, people in the center of the country, people into manufacturing, farming and other industries are hurt because over time due to work that the west and east coast people did, these tasks became commoditized and it was easier to import labour and do these things. In many cases, machinery and industrial automation (again a work of the coastal people) caused the central people to lose their livelihood.

Hence when it came time to vote, the media, the influencers and everybody jumped into it saying we need to focus on things like equality and making america more open. While the people in the center said — hey — just give our jobs back.

Now we know how that turned out — people on the coasts are less in numbers compared to the center — and if it was a voting game, the center had to win.

But the deeper problem is that we are attacking the symptom directly. Here is the chain of thought:

1. I lost my job.

2. I want my job back.

3. This presidential candidate keeps saying he’ll get my job back.

4. Give me my job back.

5. Who took away my job? That immigrant!

6. They do not belong here, send them back.

7. Give me my job back!

Now I don’t mean that in jest — someone losing their job is serious and I sympathesize with the situation, but again, we are attacking the symptom instead of solving the root of the problem.

Currently the proposed solution is to force companies to setup manufacturing bases back in the US. This sounds familiar — this is exactly what China got really good at in the past decade — competing with US for crude oil resources along the way. Have we come to a point that US political strategy is guided by what China has been doing over the past few years? Even India is heavily focused on bringing manufacturing into the country with the “Make in India” program.

So where does this leave the current state of the economy?

Most computer and vehicle manufacturers will setup a triplet of bases — US, India and China. For example, it is rumoured that the other two sites for Tesla’s battery megafactories will be China and India. I don’t think this will bring any economic benefit to the US. To really understand who gets the lion’s share of the value chain, we need to understand where the design is happening.

This is not because of the number of people design oriented jobs employ, but because the wave of manufacturing automation is going to be led by the industry that can pioneer automation.

If the United States is going to be distracted by bringing jobs back to the country by forcing manufacturing back into the country, they might lose their current status as design leads and eventual manufacturing automation pioneers — the challenge being that they would bring manufacturing back into the country, but run by robots instead of humans.

Marijuana was considered illegal by most major countries and governments actively fought to educate the people about its supposed harmful effects and how they were protecting them.

People used to get very touchy on topics of racism especially in the white/black categorisation. Also women who called themselves feminists would get very touchy if someone made a comment that put classified women differently from men.

Censorship was at an all time because of how easy it was. All broadcast media like TV and music was subject to extensive censorship of words even though everyone knew what was being spoken. As if everyone was playing along in the game of acting like a few words did not exist.

Governments of different countries used to treat their own currencies as something unique and every other country was a separate distinct thing. You had to pay large fees to convert between these different debt certificates that the government would give you.

People used to walk into secure locations to press a button to give their vote to a representative who would then act on their behalf. At that time, such a system existed because it was believed that not everyone can actually participate in the decision making process of the government and hence would elect a chain of representatives

At its peak, the realty business for greedy and went bust. But it came back because governments were afraid of the chain reaction it might cause to let capitalism work they it is supposed to work. Hence the illness and virus continue to live for a few more years. But of course it was no longer important to get a restaurant in a prime location because people would generally simply order in. Even if they did eat out, it would be something closer to where they live.

People’s choice of where they live would no longer be a function of where they worked. Office was relegated to not being a place of work, but a place of collaboration — going to office meant that something was seriously wrong and everyone needed to get into a room to figure things out.

Offices stopped trying to fight for the most exotic tech spaces as it was not very important to have an office location.

Instead of people migrating onwards to the city centre, everyone started spreading out. Cities grew in area and smaller cities became larger. Population that traditionally used to concentrate towards dense areas started to instead radiate outwards into less dense areas — like a gas molecules diffusing into vacuum

This essay is inspired by “What You Can’t Say” by Paul Graham. You can read it here: http://www.paulgraham.com/say.html

Identity

Originally we had a name that was used to identify us. If we went to a shop or to a friend we would mention our name and they would be able to mentally pull up a backfground info about us. For example what kind of person we were, could they trust us and whether they are ok lending something to us. We added out family name to our first name to be able to extend this set of background data to our families. Because we grew up in closed knit communities, it was safe to assume that a child had very similar qualities to their parents. Our address was our messaging mechanism. We would communicate with bus

Skip forward to the internet and email became the means of identification as well as messaging. Plastic money called credit cards moved from the physical domain into electronic domain with PayPal continuing to use the concept of an email address as a means of payment.

Later with rise of mobile phone apps, it was much more easier to use the phone number as a means of auth. Today, my phone number has become my unique ID in this country. More than my Aadhaar number of my PAN number — my phone number has much more data about everything that I do. Messaging, banking, login to apps everything operates based on this 10 digit number. Losing access to this number in today’s age is serious — just like getting a house robbed in the olden days.

But a new layer of infra is coming up that is set to make the concept of a phone number irrelevant.

A lot of phone calls and text messaging is moving to the internet — with rise of messaging apps. These apps have their own identification and messaging constructs. Over time they have also added payments.

Let’s talk about INdia. Here WhatsApp is the most common app that one might have. Initially Whatsapp was just an alternative to SMS, and it extended the concept of a phone number by using it as your ID on WhatsApp (compared to the earlier days of AIM usernames and BBM pins).

Over time WhatsApp added phone calls and video calls. Carriers have tried to compete with this by introducing VoLTE calls — which are essentially phone calls over 4G using an open standard.

Now WhatApp has added payments as well and I think they are just one step away from becoming core infrastructure for everything we do.

WhatsApp login would be a way to login to apps using WhatsApp. Once you are logged in, you can manage payments and messaging with that business via WhatsApp.

Over time as India becomes more government controlled, there will be more and more surveillance of money spending via Aadhar and UPI. Also cash has been demonised as demonetised.

Hence I look forward to the surge of some other form of currency that can be used for under the table transactions and currently my mind is on crypto currency

Another trend I want to keep an eye on is the rise in importance of the phone number. It is almost a universal identifier for a person even more so that their e-mail ID. And it’s much more public than any other form of ID like Aadhar or PAN.

What I mean to say is that given my phone number, there is almost everything to be found about me and all online accounts can be controlled using OTP password reset.

This is not comfortable. I foresee a rise of some other form of auth mechanism. Similar to how phone took over from email ID.

Currently I am thinking it will be some sort of crypto based thing. The phone device will generate a random token and then use that for login to all services.

This is technically a phone number itself. It starts out as a random 10 digit number but slowly becomes a very important part of or life.

That means there is something that will occur on the phone number side of things. Especially because concept of cellular call switching is going away.

This is taking me in the direction of a WhatsApp based auth system. Imagine SMS OTP auth but over WhatsApp.

Like a cross between Facebook login and SMS based OTP.

This will be more secure with longer auth codes and no need for manual typing.

WhatsApp is currently tackling payments and based on above thinking they might tackle auth next. Especially if they dig into merchant payments.

WhatsApp OAuth

Everyone likes to talk about the good old days of simply being able to create a new file with a .html extension and start making a website instead of how we do it today — setting up a highly complex build system to get anywhere near to a basic hello world project.

The HTML Era

HTML (Hyper-text Mark-up Language) itself was simply a way to mark text and specify the styling and treatment of text. You could technically publish text as is online. But you could really go places by marking up a specific part of your text as an anchor and then being able to click that and open another document!

We were not satisfied with the basic capabilities like bold and italics so we built CSS. Now, we wanted to modify some parts of the HTML/CSS in response to things like clicking things, so we implemented a scripting language to quickly specify such relations and have then run within the browser itself instead of a round trip to the server.

But the key point here is that the way we developed our site was still about having multiple files open and writing the HTML/CSS and JS separately. Even deployment extended further by simply hosting these files on a static file server like Apache.

The PHP Era

Overtime, we were still not happy with the ability to markup text (HTML), ability to style this markup (CSS) and even be able to script small behaviours to this (JS) — we wanted to serve a different HTML to different people!

And so was born PHP, it feels like a natural extension to HTML itself. You write your code between your HTML file itself and then be able to run those parts on the server, which further generate HTML and the final HTML gets send to the browser.

This was extremely powerful. We could serve completely different pages to different users even though all of them access the same URL like Facebook. We could use a database on a server and store some data there, then based on some conditions use this data to modify the generated HTML and technically have an infinite number of pages available to serve (e-commerce).

The Dark Age

Somewhere on this path to render pages on the fly (SSR) and render pages on the client (SPA) we forgot about the performance of our webpages. We were trying to build apps. But the web is about presenting content first and foremost!

We built large front-end frameworks to do everything on the client itself. But then we also ported those JS engines to the server so that we could run the same code on the server as well.

We built complex build and language transpilation tools to make sure that the differences of so many environments are abstracted away so that we can run the same code everywhere.

On the one hand we were stuck scaling these servers while on the other hand we were showing users blank white pages till all the JS had downloaded and they were ready to see the content.

To solve this, we decided to do both! We would render the page on the server first, then serve the JS, then the JS would take over and run on the client only. Nice! Congrats — you have now shipped your server-side code to each and every user for them to run it in their browser.

Even if a small part of your page is interactive, the logic to render the entire page is shipped to the client and any subsequent pages that would render.

The JAM Stack

Javascript, APIs and Markup — this stack is all about finding middleground from the chaos of SSR+SPA. It is about stepping back and asking yourself, what parts of my page change and what parts don’t change?

The parts that don’t change often are pre-rendered on the server and saved to static HTML files. Anything else is implemented in JS and run on the client using API calls.

This has the benefit of avoiding too much data transfer (like the hydration data for SSR) and finding a good tradeoff to shipping content on the web.

But even more importantly, this allows you to leverage the power and cost oF CDNs to effectively serve your content. And with serverless apps, your APIs will never need a server to SSH into and manage.

Conclusion

I find it fascinating that we are back to generating separate HTML/CSS and JS files and then putting them on a static file server — the CDN. It has been a decade long effort and as we come back to where we started, I feel like we are at a whole another level (a spiral?).

At the end of the day, the output remains the same, but we have much more sophisticated tooling similar to many other things in life. Imagine the complexity involved in converting your voice into bits and transmitting it over a combination of wires and air spectrum to reach the other person contrary to the past where we would just travel and meet people. Over time the outcome remains the same, but by introducing complexity and then effectively abstracting it away, we are able to accomplish things indistinguisable from magic.

I wrote this for my newsletter:

Sign up now

Photo by Jilbert Ebrahimi on Unsplash

Being wrong is better than staying shut in the pursuit of the right.

I take joy in being wrong sometimes — it’s my gateway to coming out as right eventually. By being wrong, I put myself in a vulnerable position and hence at the receiving end of a lot of information — more than I would be able to find if I had instead chosen to claim to be right from the start.

Intentionally saying something that I know has a high chance of being wrong allows others to immediately jump to correcting me and hence fill in the gaps in my understanding in the most efficient way possible. By being so visibly and obviously wrong, the amount of effort someone needs to correct me with the exact piece of missing information is trivial.

Assume you don’t know how to make the nose.

Let’s speak with a metaphor — say you are building a sculpture but you don’t know how to make a human nose on your otherwise beautiful sculpture.

You have two options:

1. Not make a nose at all because you were busy asking for help.

2. Make it in the best way you can. But you can immediately see that it’s wrong.

Now consider the difference between the two:

1. In the first option you may reach out to the best sculptors on how to correctly make a nose and they get back to you saying they do not have time to help you.

2. In the second option, people like your sculpture but can’t bare to see the crooked nose and immediately offer tips on how to correct it or even offer to correct it for you.

Human phsychology has a basic rule — it’s always easier to edit something than create something. This is why so many writers struggle with a blank piece of paper, artists struggle with a blank canvas and programmers struggle with a blank editor screen.

Extending from this basic concept, it is always easier for someone to correct you than actually answer your question from scratch.

It’s not that you are being sinister here. Asking a question is ineffective because the other person has no context of how much you understand — they need to lay out the context for you first. But by simply saying what you know, it is easy to point out which assumptions are wrong and will eventually help you be right.

Hence, while a lot of people out there are concerned with being always right and never wrong — I believe a meaningful approach is focussing on speaking what you know. Others will instinctively fill in the gaps for you.

It’s about starting from scratch vs. starting with a hypothesis. By always having something on the table you give others the option to shape the discussion and everyone learns.

Next time you are in a discussion, don’t be afraid of contributing the nose to the sculpture on the table —even if you immediately know you did it wrong, someone will fix it for you, but if you never contribute then it might never exist.

<a href="https://medium.com/media/76db180fa6aefb00e9f9b0371c0f5ba8/href">https://medium.com/media/76db180fa6aefb00e9f9b0371c0f5ba8/href</a>

(source)

Scraping and aggregating millions of candidate resumes and profiles is a solved problem. Storage is free. It’s time to bring meaning to each profile.

In the 90’s anyone could create a website. You could either use one of the free website creation tools, or you could create one by hand after learning some HTML/CSS if you were so inclined. You would still manage to get hosting for free using the many free hosting providers. It was suddenly possible for domain experts to build a community around themselves. Low-carb cooking, acrylic-on-glass painting, vintage motorcycle repair — you name it and you could build a community around it.

But with the proliferation of these islands of information, arose a new problem — how do you get to these people? Yahoo solved this by curating a database of these websites by category. Looking for info about that particular cooking recipe? Search the Yahoo Directory and you shall find it.

The Yahoo Small Business directory.

As the number of sites in each category grew, we now had to find out which is the best of these? Everyone is claiming that they are a great cooking recipe website — but which is truly the best? Which one deserves your time?

The early days of Google.

Here comes Google and their PageRank algorithm. By ranking each page by the number of incoming links as well as scraping the site for content — Google was able to pair the claims of the website with the validation of other similar websites linking to them. It was the first verified index of interest groups on the web.

The story is not so different for candidate resumes and the problem of hiring.

Initially the resume was a closely guarded secret. If you were a recruiter I trust, I would share a hard copy of my resume with you, which was a history of my past companies and roles. This would still not contain details about my specific project experience and you would need to convince me for me to share those with you over phone.

Eventually the resume moved online, and you had the concept of a candidate profile online on multiple job boards. There are multiple online networks of candidates to tap into. Each candidate keeps their profile up to date and we can easily search by the kind of skill we are looking for. We quickly queue up a lot of candidates from these portals in our applicant tracking tool and then begins the arduous process of evaluating them.

To me, this sounds similar to having come across a large list of relevant sites and now visiting each of them to actually find the website that is most likely to answer my question.

Is there a way to better validate candidates and really know who is the best — even before we start interviewing?

The answer is that this too is a solved problem — but with a catch. Companies interview candidates, and keep an ongoing record of the good ones, even though they might not have an opening at that moment. Larger companies have larger records, but there is still no way to tap into this hidden secret. This is a closely guarded secret just like the resume was in the early days.

I believe it is a matter of time before we are able to tap into a validated pool of candidates — where we can jump straight to the interview process without needing a review or screening. This is possible by running machine learning on the feedback that an interviewer might post and building a profile of what the candidate is good at. The feedback remains private to the interviewer and the company, but the validation of the candidate’s skill can be tagged to the candidate profile.

It’s time to move from a candidate database to a smart searchable pool of verified candidate profiles.

Instead of having a candidate profile with ten different skills listed — and god forbid the all-too-common skill of being able to use Word and Excel — we have a resume of just three skills that the candidate is really good at — because they have already been interviewed on them and were positively rated.

In this world, when I am looking for candidates who are great at building websites, I instantly see the ten best candidates suites for the role, and I can skip the review or screening stages with confidence — and directly call them in for interviews!

Now, that definitely sounds like how hiring should be today!

We are building Caliber — a system that understands candidates and openings more deeply. You can try our first product today — a single gathering place of all ongoing evaluations for your team.

Caliber

Sign up is free and takes just a single click. Try it out!

We are also on Product Hunt:

<a href="https://medium.com/media/410f56d3a49e35c407ff1a2dbfb5fb86/href">https://medium.com/media/410f56d3a49e35c407ff1a2dbfb5fb86/href</a>

The Era of Candidate Databases is Over was originally published in Caliber on Medium, where people are continuing the conversation by highlighting and responding to this story.

by HasGeek

At JSFoo 2013, a Javascript conference in Bangalore — in a turn of events, I gave a small 5 minute flash talk with my friend Chetan Agrawal.

Today in a world of web-apps, everything runs in the browser, but that means that the web-app itself needs to load every time you access the site. How about the server render it the first time so that you can see the content straight-away?

<a href="https://medium.com/media/45226a93e55704448e0e1a80a7f6fd6b/href">https://medium.com/media/45226a93e55704448e0e1a80a7f6fd6b/href</a>

Head on to this YouTube video. We are at the 6min 35sec spot.

At MetaRefresh 2014, I gave a small 3min flash talk about a little project of mine. MetaRefresh is a conference for user interfaces on the web. It’s a yearly event in Bangalore.

Front-end engineers are in a position of power today. Node.js, browser-side frameworks like Angular/React, back-end services like Pusher/Parse/Firebase and mobile app platforms like PhoneGap — basically mean that a front-end engineer can build an entire app, make it realtime and also make it work on all mobile devices. This is insane!

Head over to my little talk on YouTube.

<a href="https://medium.com/media/408bd7c72a8023243d377c679743d59e/href">https://medium.com/media/408bd7c72a8023243d377c679743d59e/href</a>

Ira Glass gave an interview in 2009 about storytelling and happened to mention something that absolutely nobody tells beginners — without which many of them just quit.

In 2011, David Shiyang Liu made this kinetic typography and it has one and half million views today. He made it as part of his 30 day creativity challenge, and he came up with this on Day 6 — after, having worked for three days on it.

ZEN PENCILS

_Ira Glass (1959-) is a radio broadcaster and host of the award-winning program and podcast This American Life . My…_zenpencils.com

In 2012, Gavin Aung Than of Zen Pencils made this beautiful cartoon. He mentions how the very thought of starting Zen Pencils after quitting his job took around having made 500 cartoons first.

In 2014, Daniel Sax made this beautiful adaptation and has garnered another one and a half million views. But success didn’t come easy — it took him a year to go from concept to upload.

In 2015, young creative motion graphics artist Saar Oz made this a year after Daniel’s video. This one is going to touch a million views. But it is actually his 12th motion graphic video before he had this huge hit.

You just have to fight your way through.

The message that Ira makes is beautiful, but I love how the message shows in two ways in all the above works. One literally by what shows on screen, and one metaphorically by what is behind the work.

The original video is just one clip of a four part interview. Ira mentions that one needs to constantly keep doing till the really great things emerge. And yes, to come up with this little gem of advice, it took an entire twenty minutes speaking on the interview.

It takes time, so the best you can do, is to just do.

If you enjoyed this little compendium, please hit Recommend — it helps other people discover it on Medium.

Today, John Gruber wrote About That Purported 10.5-Inch iPad, which makes good points against the rumors spreading. Here, I demonstrate how the line-up might evolve with pictures.

This month, we’ll see a silent update to the iPad line-up, without an announcement event. All iPads get a $100 price-cut as always.

Notice the new $199 entry-point price for iPad is now possible with the iPad mini 2 capable of offering a great experience and might become the go-to tablet for kids instead of the iPod touch.

This would mean no new iPads for now, but the pricing across the line-up is very tempting. Notice the iPad Pro 9.7” sits in the sweet spot $499 price-point that iPad used to always be at. It is also the most capable device in the line-up.

But, look at what happens next year:

Apple introduces iPad Pro 10.5” in either October 2017 or March 2018. Now the iPad Pro 10.5” does to the iPad mini what the iPad Pro 12.9” did to the iPad Air. It takes the number of pixels and doubles them. Hence it is two iPad minis stacked side by side.

This means that the iPad Pro 12.9” and the iPad Pro 10.5” have the exact same resolution — just like the iPad Air and iPad mini have always had. Only the pixel densities are different.

The line-up has this sense of purpose now:

1. Air/mini are the mainstream models with the ability to run one full size app at a time in portrait.

2. Pro models run two full size apps while in landscape, with support for Apple Pencil.

In both cases, the difference in the device size is only a function of the pixel density and not the actual pixels.

Do you agree? Does this seem like something Apple might do? Let me know in a response below, otherwise please hit Recommend!

“Mr Wilstkirkson.”

“Oh, dear boy. Why don’t you call me by my shorter name, Willy, like everyone does?”

“Because it’s your name.”

“But, its so long. I use Willy everywhere possible.”

“Your tea is ready Mr. Wilstkirkson.” said Roy, as he didn’t find it a reasonable discussion.

“Ah, let it be, you will get bored someday. Please keep it on my study. I need to make a phone call.”

Roy kept the tea cup on the study. But not just keep it, he placed it. As in, meaningfully, at a certain spot on the table. He thought that placing it in the center didn’t look good. Placing it to the sides would risk the cup falling. So he placed it towards the right, but not near the edge, thinking that Mr. Wilstkirkson drinks his tea with his right hand so the placement was meaningful. He then went off to wash the dishes; the cup placed on the table like it was a still life photography subject or a museum artifact, but only steaming and live.

Roy had always been like this. He thought about everything, tried to find meaning in everything. He had been working with Mr. Wilstkirkson for a year now. He loved the job, as Mr. Wilstkirkson was mostly out of the house and had no family, there was not much to do and a good pay. It gave him time to think. And that’s what he did. He had reached the kitchen, and started washing the dishes.

He wondered why Mr. Wilstkirkson wasn’t drinking the tea while talking on the phone. He weighed in the benefits and the problems. He felt that it was better because it wouldn’t cool off. Also it would keep the throat wet while talking. The only issue he found with it was if he had to speak just after he took a sip. But he quickly dismissed this issue, by concluding that he could have a sip when the other person was talking.

Involuntarily, his mind had made note of the conclusion and he thought that he would tell Mr Wilstkirkson about it when possible.

As soon as he had finished washing his first dish, he heard Mr. Wilstkirkson speaking in the other room.

“It was really nice speaking to you, Mrs. Mirson. I am glad you took the time to address my concern. It has been so kind of you.”

Roy’s mind was racing again. He wondered what Mr. Wilstkirkson actually wanted to say out of all this. He felt that this was just not meaningful. It was not what Mr. Wilstkirkson really wanted to say. He was so sure about this, because Mr. Wilstkirkson was the landlord and Mrs. Mirson and her family had not payed their rent for three months now. He made a mental note to try to understand this from Mr. Wilstkirkson some time. He had to make sense of why people did what they did. Try to understand why things happened the way they happened. It had always been important to do so.

“Yes?”, said Mr. Wilstkirkson.

Roy realized that he had come out into the other room, where Mr. Wilstkirkson was talking on the phone. He thought why he was there. He tried remembering something he knew he had remembered to remember.

“Mr. Wilstkirkson,” he finally said after he was sure that he had just one important thing to ask. “Why do people have tea in a cup?”

Mr. Wilstkirkson was confused. “Oh, yes, my tea. Thanks for reminding. You kept it on my study right? Please get it here.” He was always astonished by these sudden weird questions that Roy put forward, but he had believed that he was just joking and never really paid attention. Roy had been a very loyal servant and he was happy to have him. He trusted him. Roy always seemed lost in his thoughts, but that had never been much of a problem. Except the time he burnt a precious shirt while ironing it. Or when he dropped the Egyptian tea cup thinking that he had handed it over. Or… But he was just fine.

Roy remembered the tea he had placed on the study and went off to get it, forgetting the question at hand. It was common with him. He had lots to think about, but never did he really remember any of it. it just came and went. Kept his mind busy. He never was idle. At least not mentally. Sitting at the bus stop, he noticed how people spoke on the phone, how they reacted when the bus arrived and how they passed time while they waited. He was busy. There was much to think about. He was proud of himself for this. He was a good observer and he noticed things. That was what was important to him.

This story is a metaphor about focus, about being observant, about doing only what matters to you — and then doing all of those together, living each moment deeply and fully.

While replicating the End-to-End Deep Learning approach for Self- Driving Cars, I was frustrated by the lack of visibility into what the network is seeing. I built a tool to fix this.

The simulator and the python script running the neural network communicate over a websocket connection. I decided to write a small browser app that would also connect to the python script over another websocket and collect some additional data that I can visualize.

Here is what I was able to come up with:

On the right we have our Udacity Simulator running. On the left is my little React app that is visualizing all the outputs of the convolutional layers in my neural network.

As you can see, it runs in realtime along with controlling the car. It was a good exercise, but I have my doubts on the usefulness of this. It is still not clear how the network is making its decisions based on these.

You can find the code here:

paramaggarwal/CarND-Behavioral-Cloning-P3

_CarND-Behavioral-Cloning-P3 - Starting files for the CarND behavioral cloning project (P3)_github.com

This article is a work in progress on the overall goal of gaining better visibility into neural networks. Let me know your thoughts in the comments on what else I could try towards this goal!

We open them up and look inside at some of the electronics and marvel at today’s age of ubiquitous wireless earphones!

I have been wanting to try some lightweight bluetooth earphones since a while, but they have always been in the $99 range and higher. Recently I came across these on AliExpress and they had really good ratings. So I gave them a shot.

They were just $ 6, inclusive of shipping, so I ordered them. I’m glad I did! They are simply amazing. They do their job well and the audio is decent. My main take-away here was that I was looking at wireless earphones all wrong.

Till you try them out, our assumption is that wireless earphones will simply cut out the wires — so what? But it turns out that it makes a world of difference.

For example,

1. I don’t need to think twice before standing up from the desk.

2. If I am watching something on the phone, I don’t have this cable dangling from the phone and I am able to hold my phone more comfortably.

3. I’m able to keep the phone aside for a bit without having to worry about yanking earphones out of my ear.

It feels like we were tied to a leash and somebody just removed it!

Let’s look inside.

That’s it! Just one chip. Notice the fray wires going into the left and right earphones. The micro-usb port along with the tiny charging IC beside it. And the antenna on the bottom right for Bluetooth — etched right into the PCB itself for cost and simplicity!

Even the battery is really tiny, the size of this board itself. It boggles my mind about the size and cost of today’s electronics! Companies charging above $99 for such earphones are not doing so because of the cost of the parts, but because it is hard to make really great earphones!

Things like wireless performance, audio resonance and battery life are challenging problems. It is easy to put together a working wireless earphone today, but really hard to make great ones.

The back side of the PCB is just buttons and the tiny black microphone. Notice the cutout for the antenna on the other side. So much going on in so little space.

So there you have it! A quick look at where we are today in terms of Bluetooth earphones. Cheap, small and awesome!

I would recommend getting these here and trying them out. It’s just $6 incl. of shipping on AliExpress.

This technical post is the complete opposite of my original philosophical post about my experience with this project.

The Problem

Given a dataset of 30,000 traffic sign images with dimensions of 32x32, we are supposed to build a model that can predict these among 43 different possible sign types. We’ll start from scratch with a directly connected model, slowly introducing hidden layers and then eventually use a Convolution Neural Network. We’ll visualize how each change will affect our accuracy rates.

Data Pre-processing

It is helpful to perform some mathematical operations on the input data to have it work well with the rest of the process.

One-hot Encoding

We generally encode the labels into an array with only one element set to 1, this basically says that the probability of that occuring is 1, while others is 0. We pair it with the tf.nn.softmax_cross_entropy_with_logits() method.

But, since TensorFlow 0.8, if we have a unique label for each label, then we can instead use tf.nn.sparse_softmax_cross_entropy_with_logits() which directly takes our labels array and is faster.

Histogram Equalization

A typical photo has the colours and brightness limited to a certain range. We can use this method to equalize the brightness levels of photos. I like to convert the image to grayscale and then apply a histogram normalization.

import cv2 bw = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) equ = cv2.equalizeHist(bw) ```![Visualization of black and white and histogram equalized versions compared to original.](/img/1*pHyGGDIGjOryJFyHHAB9EQ.png) #### Normalisation Generally we normalize the input colour values down to -0.5 to +0.5 compared to the original of 0 to 255. Here, it is more important that the mean be around 0. It is ok to go from -1 to +1 also. Don’t do `x_train = (x_train — 128.0) / 255.0`, as this is not as good as doing `(x_train / 255.0) — 0.5`. Very subtle difference, but makes much more sense. You can even go ahead and do `(x_train — np.mean(x_train)) / 255.0` if you are feeling adventurous. With more trial and error, I saw best performance by letting the values run between -128 and +128. Hence, my solution simply does `x_train — np.mean(x_train)`to normalize the data. I believe this is related to losing fidelity because of forcing the input to lie in a very tiny range of -0.5 to 0.5. Let me know if you have experienced otherwise. I think the key is to have a mean close to 0. Regarding min and max values I have seen different people recommend different things so you should try some variations. ### Data Augmentation There are three reasons to generate more data by augmenting it. 1.  We want to have more data to train with, this always helps. 2.  We want data with variations on rotation and translation so that the model can generalize better and be agnostic to specific positioning of the sign. 3.  Also, certain classes are under-represented in the data, by augmenting and adding data, we can give more samples to these minorities. ![](/img/1*oHPth2wN-CJzbVwIPqKdSA.png)![We want to equalize the representation of classes in our data. \[Before/After\]](/img/1*X3waH4jxI6IFy-ChDoucjg.png) #### Translation I was initially translating the images within +-5 pixels which I later learned were too much for the capabilities of my model. We need much larger convolution layers if we want to make it very agnostic to position of the sign. Currently, the model expects a cropped version of the sign, so I only translate +-2 pixels, which is still quite a bit given image sizes of 32x32.

scipy.ndimage.interpolation.shift(image, [random.randrange(-2, 2), random.randrange(-2, 2), 0])

#### Rotation A similar realization struck me once I visualized my augmentations. I was rotating +-20 degrees, which is too much! I eventually reduced this to just be +-10 degrees.

scipy.ndimage.interpolation.rotate(image, random.randrange(-10, 10), reshape=False)

### Neural Network This is what we are here for! Let’s get to the main course. I logged the performance of my network for each change I made, now I get to share it with you! ![Before I corrected my over-augmentation of data, this is what I started with.](/img/1*5wypraMewwdNUOT39d7oeg.png) First let’s look at my experimentation with having a basic neural network with just a single hidden layer. As you can see above, even flattening and connecting the image data to output directly gives you are baseline of 70%! Once we add a hidden layer, we start overfitting — even a dropout doesn’t help. By continuously reducing the width of the hidden layer, we get good results at around 48 neurons. I brief experimentation with adding another layer also didn’t so much benefit. While I was doing this, I added visualisation to my augmented data and realized that I was translating and rotating them a bit too much. After being more conservative while augmenting the images, we begin our journey. ![After correcting the over-augmentation, we begin to see nice accuracy, with very basic networks!](/img/1*-vi19j9WZrFBV002nFhuVQ.jpeg) You can immediately see that the directly connected model jumps from 70% to 85% after being more conservative about augmenting the data. I quickly try the variation with a single hidden layer and we are already above 90% — this is going great! I also experiment with going overboard on the hidden layer, but keeping it in control with a dropout of 0.5 — this also gives good results. But trying to add one more hidden layer wasn’t very helpful. At this point, we now look towards adding convolution layers to our model to try to move into the 95% territory. This is when I realized I stepped on burning coal. ![Trying to get the convolution layer depths right.](/img/1*EZVCrx26s-ImqNH3MwT2Pw.png) First I tried out different depths for the convolution layers. I noticed that smaller depths actually helped the network. ![Making the fully connected layers work right.](/img/1*G8sWDM_Hz2OTfuMpWEL3GA.png) Here again, I learned that smaller layer width actually helped the network. If the layer width gets too high, we could try to improve it by using dropouts, but a smaller network is probably going to generalize better than a large one with dropouts. ![I think I want to die.](/img/1*qQuPiqjLN3J7C6GrOxpApA.png) I went crazy by this point, nothing I would do would push me into the 90% range. I wanted to cry. A basic linearly connected model was giving me 85% and here I am using the latest hotness of convolution layers and not able to match. I took a nap. The next day, I approached this differently. I tried to run my network without my pre-processing and augmentation steps, and it did better on a standard LeNet model. I tweaked my normalization steps and realized that forcing input to be in -0.5 to +0.5 range was actually hurting it a lot! So I immediately made it only be in the -128 to 128 range and I was quickly friends with the 90% circle. Success! ![](/img/1*EnrXaMtun9m5Hv8zeqw3Gw.png) Here is my final model. I am surprised how tiny it is! A small amount of dropout adds redundancy to the fully connected layers making an interpretation of the input. Again, the widest this network goes is 128 — that’s so awesome! ### Visualization Use matplotlib’s `subplot()` function to be able to render images side by side for comparison. This helped me a lot in being able to make grid’s of visualizations for comparing augmented versions of images. Also, visualize the confusion matrix on the test set. Notice that the model is weak at learning the classes in the 0 to 10 range. With this understanding, we can dig deeper and maybe add more training data in that dataset. ![](/img/1*4pQZhNJ7_NfkXXs2yQxFbw.png) Pick out some random images from your test set and visualize the output of the neural network. What we are looking for are singular peaks that say that the model is confident about a particular class. Unfortunately, our model can’t decide what it wants for lunch. In most cases, it is predicting upto 3 different classes for each input. Very indecisive. ![](/img/1*uHcYtfdi0lbMkk1IVTfmHg.png) While I haven’t tried it out yet, use of TensorBoard is recommended. I look forward to using it in a future project. This will help us visualize the weights while training as well as the convolution filters. Display the time taken for each Epoch to understand the complexity of your network. My network just takes 5 seconds on an AWS GPU and 35 seconds on my quad-core MacBook Pro. Use smaller epoch counts when tuning, as you can get a general idea of the performance within the first three epochs. ### Conclusion Made it alive on the other side of the burning coal. My feet hurt, but I am better prepared to face neural networks now. No fear. I have shared the code and model on GitHub: [**paramaggarwal/CarND-Traffic-Sign-Classifier-Project**   _CarND-Traffic-Sign-Classifier-Project - Classify Traffic Signs._github.com](https://github.com/paramaggarwal/CarND-Traffic-Sign-Classifier-Project "https://github.com/paramaggarwal/CarND-Traffic-Sign-Classifier-Project")[](https://github.com/paramaggarwal/CarND-Traffic-Sign-Classifier-Project) --- _Much thanks to_ [_Mehdi Sqalli_](https://medium.com/@MSqalli)_’s_ [_project code_](https://github.com/MehdiSv/TrafficSignsRecognition/blob/master/final_Traffic_Signs_Recognition.ipynb)_,_ [_Vivek Yadav_](https://medium.com/@vivek.yadav)_’s_ [_project code_](https://github.com/vxy10/p2-TrafficSigns/blob/master/Traffic_signs_Col_val_final2.ipynb) _and_ [_Jeremy Shannon_](https://medium.com/@jeremyeshannon)_’s_ [_project code_](https://github.com/jeremy-shannon/CarND-Traffic-Sign-Classifier-Project/blob/master/Traffic_Sign_Classifier.ipynb) _for this project that guided me to do better and inspired me to share my learnings like they did._ export const _frontmatter = {"title":"Intricacies of Traffic Sign Classification with TensorFlow","description":"This technical post is the complete opposite of my original philosophical post about my experience with this project. Given a dataset of 30,000 traffic sign images with dimensions of 32x32, we are…","date":"2017-01-31T09:36:39.833Z","categories":["Machine Learning","Neural Networks","Deep Learning","Artificial Intelligence","Self Driving Cars"],"published":true,"canonicalLink":"https://medium.com/computer-car/intricacies-of-traffic-sign-classification-with-tensorflow-8f994b1c8ba"}

That means me doing a lousy job at building something can pay someone else’s salary who later fixes it. My incentive in turn, is also to only fix issues and it is a vicious cycle. I’m not thinking of building solutions that make the problems irrelevant.

Is there a better way around, than actually fixing them? Yes, to disrupt the problem. To make it irrelevant.

Sounds familiar? Not in the domain of fixing a company’s software. Or maybe think again. Disruption redesigns systems. Only these systems are at a human level. They change society. They make entire industries irrelevant.

Similarly redesigning a company’s systems can disrupt the existing problems. A popup window being bug prone and having compatibility issues. But hey, does that popup window even need to be there?

Isn’t redesigning bad, you say? You wouldn’t redesign the wheel. But that’s where the misunderstanding lies. You would not reinvent the wheel, but you could redesign a particular aspect of your factory’s gear system.

Design is thinking. And when you think about something deeply, you ask questions. You think how. You think what. You even begin to question why.

Rethinking might not mean doing it all over again it means understanding what you already have and appreciating what works and what doesn’t. Then you attempt to amplify your understanding.

You improve the essentials and discard the inessential. Note, this is different from fixing issues, because fixing issues only involves the knowledge of the issue itself.

So next time you see a problem, wonder does the problem even deserve to exist. Fixing an issue in the system is like applying patches. Till when?