201

It is the first time that I am doing HR, and we are looking for a developer. The selection process is three rounds: technical phone interview, programming task (0.5 - 1 hr challenge), and then finally an interview with upper management and me.

The problem that I am having is that when I give some candidates (mostly fresh grads with 1 or 2 technical internships already under their belt) the programming task, they not only do not complete it in the given time frame (a week), but I don't hear from them again unless I follow up.

I'm thinking about ditching the programming task, but I really think it can help establish who really knows the languages listed on their resume.

How can I improve our hiring process?

UPDATE SINCE THIS QUESTION HAS BEEN POSTED

For the most part a lot of candidates are put off by the programming tests, which is very frustrating since I need to go through A LOT of candidates to find one that is willing to do it. With that said, I have found some willing to do them, and generally have found that:

a) It shows that they have a good attitude towards the company and role if they are prepared to go the extra mile to complete it.

b) They have programming ability. Sure they may have cheated but if they have attempted it, again, a good attitude for us is much more important since they will be much more easily manageable.

I have since hired a graduate developer, who attempted and completed the exercise successfully. He also completed all the additional 'bonus points' on the task. It is too early to say how good he is at his role, since he has only recently started.

On some occasions I do avoid giving the exercise to developers if they already have a strong portfolio working for good brands, and a track record. Since to get into the large brands he would have to do their own entrance tests.

SECOND UPDATE SINCE THIS QUESTION HAS BEEN POSTED

The graduate developer has turned out to be a star employee. We have kept him on and gave him a payrise.

bobo2000
  • 12,030
  • 15
  • 46
  • 81
  • 96
    If somebody doesn't have the wherewithal to complete a task for an interview, I wouldn't expect them to complete tasks as an employee. It would be different if you got feedback saying that the task is too difficult for the provided time frame, but I take it they are not giving feedback at all. What country are you in/taking candidates from? Either way I don't believe removing a test to employment is a valid path forward in this case, sounds like you are chasing the wrong candidates. – Ron Beyer Feb 02 '16 at 14:49
  • 230
    Speaking from my own experience as an entry-level dev, the demand is so high that I can skip any interview process with a online programming test and easily end up with a good offer (note: not technical interview with programming questions, which you should do, I'm referring to a 2-4h test done during my own time after which I may or may not ever hear back) Suffice to say demand for actually experienced people is probably even higher. Why jump through your hoops when they get recruiter emails every week, if not every day? – Cat'r'pillar Feb 02 '16 at 15:34
  • 149
    I avoid interviews with programming tasks, unless the job is above average. I see no need in losing a few hours to a company I'm interviewing for, considering that I can get plenty of interviews without tests. – undefined Feb 02 '16 at 18:11
  • 2
    Comments are not for extended discussion; this conversation has been moved to chat. – Jane S Feb 02 '16 at 23:00
  • 2
    Let us continue this discussion in chat. –  Feb 03 '16 at 16:06
  • 28
    Trust me, you want to be scaring away candidates who cannot program. You may want to revise its content and/or your approach for presenting it (for instance, maybe invite them for an onsite interview, pair them with a programmer, and have them sit in a room for an hour with a workstation and pair-program the exercise). But getting rid of it entirely is a very bad idea. – aroth Feb 04 '16 at 04:18
  • 17
    Personally, I see not having candidates code as part of their interview as a red flag. If potential future colleagues got into the workplace without showing any technical skill and I'm going to help maintain their code, that could be bad times. Additionally, this is one of the points on the Joel Test. – FreeAsInBeer Feb 04 '16 at 05:01
  • I believe that the source of your candidates is critical. Recruitment agents are interested in the easy way out. Locally, they're paid (hansomly) to get people off of social security payments, so that's where they concentrate their efforts. If you're not getting SS, they're simply not interested (Quote "Don't waste your time, don't waste our time".) Consequently, my degree and 40 years' experience is never presented by recruiters to potential employers - and most employers insist on using recruiters exclusively. – Magoo Feb 04 '16 at 06:01
  • 1
    This blog post on "technical interviews make me cry" from an ex-google now khan academy developer thinks that a task that you're given a week to do is by far the best way of doing technical interviews. – icc97 Feb 04 '16 at 09:30
  • 3
    @FreeAsInBeer I am doing exactly opposite. I believe that employee can't be judged unless he does some meaningful job, so I'd rather join a company which hires for trial period rather then be stuck with someone un-firable because "he excelled some abstract test, so he must be good at completely unrelated real-life scenarios". Especially what I look for in a junior is that his code is easy to read and maintainable by the rest of the team, not that he solves the P-vs-NP problem on his first day on the job. – Agent_L Feb 04 '16 at 10:49
  • 6
    @Agent_L yes having a probation period is always a good idea, and we do that here. The trouble is that by the time you have found out that they are not any good, you would have spent 3-4 months worth of salary and lost time. Is it worth it? – bobo2000 Feb 04 '16 at 11:05
  • 3
    What does a code challenge tell you about a candidate? Nothing. Even if you're doing it in the office as part of a regular interview you're either going to a) present them with a trivial problem that tells you nothing or b) you're presenting them with a problem that can not be reasonably completed within a few minutes unless the candidate has seen it before. Which, again, tells you nothing. "Determine if a linked list contains a loop" is one such "gotcha" question that is inappropriate and stumped comp sci PHD holders for 30 years, but the solution is simple once known. – Draco18s no longer trusts SE Feb 04 '16 at 18:20
  • 3
    You're not following this by any chance, are you? In an ideal world I'm sure it's the perfect way to hire programmers from an employer's perspective but most talented programmers will be gone before #5. No one wants their time wasted on something that is not guaranteed (and let's not mention that in the US such a difficult entry process is ultimately undermined by the 2 weeks notice that is standard before firing someone). – Nobilis Feb 04 '16 at 19:16
  • 1
    @Cat'r'pillar because then you know that you'll be working with people who have the skills to pass the test, and that there will be somewhat of a challenge working there. Of course, if you are not that much interested in the position in the first place, there is no reason to go to any interview. – njzk2 Feb 04 '16 at 23:32
  • @Draco18s not necessarily, you can devise 'simple' tasks that use a lot of technology, to give one example, a feedback form where the data is posted to a server. That alone requires AJAX, Jquery, javascript, CSS, HTML, and JSON. If the candidate can do that, sure it is not the most complex thing ever I at least then know he knows all of the technologies above to some level which reduces risk. – bobo2000 Feb 05 '16 at 15:13
  • 2
    @bobo2000 Or you could ask the candidate directly or look at their portfolio. If they're applying to a full-stack position and have full-stack experience, what does the code challenge actually tell you that you didn't already know? – Draco18s no longer trusts SE Feb 05 '16 at 15:15
  • 1
    @Draco18s that is part of the problem, when I receive applications often they dont have a portfolio or a github account – bobo2000 Feb 05 '16 at 15:16
  • 1
    @bobo2000 Do they have a work history? Is one of their references a manager from a similar position they had previously? If "no" then they're too junior for the position you're hiring for and again, the code challenge tells you nothing. Unless you are hiring a junior developer in which case, they don't need all of those skills. – Draco18s no longer trusts SE Feb 05 '16 at 15:18
  • 1
    @Draco18s I am hiring graduates with one or two technical internships under their belt right now and to be honest, asking a past manager is not always the best approach if the applicant didn't get along with him but was otherwise an able coder. – bobo2000 Feb 05 '16 at 15:20
  • @bobo2000 Interpersonal relationships should be possible to exclude when asking about technical skill. A code challenge is still the wrong tool for the job at hand. http://www.nomachetejuggling.com/2014/06/24/the-worst-programming-interview-question/ – Draco18s no longer trusts SE Feb 05 '16 at 15:29
  • 1
    @Draco18s not all managers are ethical - I once worked with one that was pretty vindictive, which is why I take what they say with a pinch of salt. – bobo2000 Feb 05 '16 at 15:31
  • 1
    @bobo2000 If the manager is going to be vindictive, the candidate won't be including him as a reference. I mean, really. – Draco18s no longer trusts SE Feb 05 '16 at 15:36
  • 2
    @aroth but this type of task doesn't 'scare off' based on ability, it selects people who don't value their time as much and may in fact lower the quality of candidates – JamesRyan Feb 05 '16 at 15:38
  • @Draco18s if it is your only job, sometimes you don't have much choice. – bobo2000 Feb 05 '16 at 15:39
  • @bobo2000 While that may be, the code challenge still tells you nothing. There is no answer that is objectively correct to all of the people involved in reviewing the answer. – Draco18s no longer trusts SE Feb 05 '16 at 15:42
  • @JamesRyan or it selects those candidates who really want to work for you. – bobo2000 Feb 05 '16 at 16:47
  • 4
    @bobo2000 no, it does not differentiate between those who really want to work for you and those who really want to work for anyone. And besides until they actually have worked for you they don't know yet if they will like working for you, so no you are not selecting enthusiastic, happy employees this way either. – JamesRyan Feb 05 '16 at 20:15
  • 2
    You can learn 90% in 1hr of what you will in several hours. The rest you won't see until they have had a couple weeks to settle in and pick up your ways of doing things (or not). These long drawn out challenges are a waste of everyone's time and frankly rude to make people jump through hoops. Companies like facebook and google have thousands of applicants to whittle down, you don't. – JamesRyan Feb 05 '16 at 20:19
  • 2
    @RonBeyer if I as a candidate believe the task is taking too much time, I won't spend time providing feedback. How does that benefit me? Just move on to the next opportunity. Is my decision good or bad for the company? Don't know, don't care. Do companies spend a lot of time providing feedback to candidates they have decided to not proceed with? – emory Feb 05 '16 at 20:30
  • 1
    I've seen coding tests done right -- but not as a pre-interview filter pass. At one place I've worked, after 80% of the technical interviews, they put me in a room with a VM dev environment, a requirements doc and a test suite built specific to the (rather specialised) job they were hiring me for; when it was done, the engineers all had opportunity to ask about my choices. This was (1) respectful of my time (they didn't ask me to spend time until they'd sunk my own), and (2) showed effort on their part (building this tooling for a position they were filling only one). – Charles Duffy Feb 06 '16 at 01:24
  • 1
    ...when a coding test is structured that way, where the company's engineers are putting their own time into it -- and it's only presented after prior interviews -- it reads less as "we don't respect your time as a candidate", and more as "we really, really care about building a top-quality engineering team". I took that job (which was at a shop that did have a top-quality team), after passing up many other places doing the no-respect-for-your-time pre-phonecall code-test thing. – Charles Duffy Feb 06 '16 at 01:24
  • 4
    If you expect the candidate to do programming for you, pay them for their time. Just because they are looking for a job doesn't mean they don't already have a job, or that their time isn't valuable. When you think about it, you are already burning cash in terms of employee hours just to interview them. So, you are asking the candidate to 'interview' for 2-4 more hours, and yet you are saving yourself the time it would have cost you in employee hours to interview them for another 2-4 hours. Thus, paying the candidate for their hours of test coding is not spending any more money. – yetimoner Feb 07 '16 at 07:06
  • @CharlesDuffy on-site tests only work if the candidate is in the same country, if you are interviewing candidates abroad that are looking for a role in the UK it is not a practical solution. – bobo2000 Feb 07 '16 at 12:12
  • @bobo2000 remotely interviewing candidates from another country is very rare and generally only done when requiring very specialist skills. The risk that they will have trouble relocating is far too high and in most cases there is an equally good local applicant. – JamesRyan Feb 07 '16 at 14:15
  • @JamesRyan it actually isn't, many graduates in particular do technical internships abroad, and they go to the best schools. Interviewed one a couple of days who is a cambridge grad. – bobo2000 Feb 07 '16 at 14:55
  • @bobo2000 well pulling this nonsense on graduates is even worse because they don't have the experience to be testing. Also if they are a cambridge grad they would in in the UK interviewing for jobs before they finish uni, so I think you are talking rubbish. – JamesRyan Feb 07 '16 at 22:43
  • @JamesRyan many of the grads I interview have done 1 or 2 technical internships, as a matter of fact, the Cambridge grad is currently in Hong Kong doing one. He is building a RESTful API as part of his internship. – bobo2000 Feb 08 '16 at 15:08
  • 4
    Did you have to complete a 2 hour HR task to prove you were good at HR before getting hired? – DA. Feb 08 '16 at 20:21
  • @DA. It's not my job role, I am a PM. I just happen to be doing it, since there is no one else to do it....and that is besides the point, if you are a confident coder, and are interested in working for the company, what is the big deal? If anything, if they don't do it, probably weeds them out since they were never that interested in working for the company to begin with (or not that confident in their coding ability) – bobo2000 Feb 09 '16 at 14:00
  • 1
    @bobo2000 point being that most people don't require professionals do take multi-hour 'tests' to 'prove their worth'. Ideally, one gets a good picture of the candidate form interviews, resumes, portfolios, education and references. That we require developers to do this is really an odd thing in general. – DA. Feb 09 '16 at 15:08
  • @DA and what do you do if they do not have a portfolio, github account but just a CV with a bunch of unknown companies on them? How do you know that at x, y company the guy was writing well written code? – bobo2000 Feb 10 '16 at 10:32
  • 1
    @bobo2000 apply that to any profession – DA. Feb 10 '16 at 14:59
  • @DA. I just looked at a CV of a previous hire who got sacked after hearing a lot about since joining the company. He had all of the skills on paper, TDD/BDD, Ruby on rails, Rspec but when it came down to it he got fired because he was poor...the bad hire cost the company recruitment time, resources, and poor quality code which has since been rewritten. There was no screening process back then. Developers are far too well paid to not be screened. If someone applied and was poor at admin, it is less of a big deal since they are not building a digital house – bobo2000 Feb 10 '16 at 15:54
  • 1
    @bobo2000 again, though, apply that to ANY professional. How do you know a doctor is good at surgery? How do you know a pilot is good at flying planes? I'm not saying tests are bad or good. I'm just pointing out how it seems odd that Developers seem to be one of the few professions where we feel a strong need to always test them. I'd argue hiring a developer that looked good on paper but not in reality was a failing of the hiring manager who maybe didn't fully check references or give an in-depth interview. A final point is that a person can do great in a test, and still fail day-to-day metric – DA. Feb 10 '16 at 15:58
  • @DA. because a bad development team can ruin a tech start up, where the quality of the product is a key factor in its success. Without any form of testing, the developer can blag the interview; references can be biased (their friend could be giving the reference) and whilst I agree that a test is not the answer anyway, when combined with a technical phone interview you will probably get a really good idea of the candidates abilities. It is about reducing the risk of a bad hire, not eliminating it. – bobo2000 Feb 10 '16 at 17:40
  • Let us continue this discussion in chat. – DA. Feb 10 '16 at 17:41
  • 1
    Mind showing a copy of one of the examples? I'm genuinely curious about how hard it is. In any case, how do candidates submit their results? I find https://ideone.com a great way to do simple tests. You create a template with the "input", they paste in their code and provide you the source and the "output". Hold onto submissions to see if candidates are just copy-pasting from CareerCup or StackOverflow. Try breaking large 1-hour problems into 3-5 pieces so candidates don't feel like it's an all-or-nothing exam. – Cloud Feb 24 '16 at 02:04
  • @Dogbert Can't release it publicly, but the way I structure the test is to have the core task, and award the candidate bonus points for additional tasks. So for example, for the Javascript test it was along the lines of completing the test using Jquery, javascript, css, html, ajax where bonus points are awarded for completing the test in a JS framework such as AngularJS. The test itself will be a very simple functionality. I recently had an applicant do the task in AngularJS even though it was not required, very impressed by his attitude. Hired him straight away. – bobo2000 Feb 24 '16 at 10:06
  • @Dogbert I also think the main thing that I am looking for is for someone to at least attempt it since it gives me a good idea of their attitude towards the role and how much they want it. – bobo2000 Feb 24 '16 at 10:11
  • @bobo2000 Interesting that candidates wouldn't even try the test. Well, maybe it's a better filtering mechanism than you realized initially. :) – Cloud Feb 24 '16 at 17:35
  • 1
    @Dogbert having the test turned out to work well for us in the end - eventually found someone who attempted it and completed it well (doing bonus points), he has turned out to be a really good hire. He needs very little training, not a pain to manage, and is hitting the ground running. – bobo2000 Mar 08 '16 at 10:11
  • @bobo2000 Good to hear! Hope things continue to go well with this approach for your company. – Cloud Mar 08 '16 at 15:43
  • This is exactly my experience too. I do the programming test at interview, on a whiteboard, which usually takes about an hour. About 90% of candidates fail it. The remaining 10% are absolutely worth not hiring the other 10%. We do it with senior candidates too, but expect a higher standard. – Michael Shaw Sep 29 '16 at 19:39
  • 1
    The market is on the side of the experienced devs who can skip those trivial stupid tests. – SuperUberDuper Dec 07 '16 at 08:30
  • @SuperUberDuper I do not give the programming test to experienced developers if they have already proven their skills on their CV by having a strong track record working for respectable brands. It is a different story however for interns/graduates, where often they lack any commercial experience, so the key is to try and find out if they have potential and the fundamentals already in place for moulding them. It also says a lot about their attitude towards the job if they can take some time out and complete the test. – bobo2000 Dec 07 '16 at 09:38
  • @bobo2000 exactly, good points – SuperUberDuper Dec 07 '16 at 12:25
  • @RonBeyer I think you're making an unfair comparison. As an employee, you committed to doing that work, you have a contract with the company, and you get PAID for your time. But as a simple interviewee, there is no commitment to a professional relationship, either from you or from the company, either implied or explicit. Therefore refusing to do a multi-hour programming task as part of an interview is very very different from an employee refusing something similar. – Radu Murzea Feb 03 '17 at 08:25
  • @RaduMurzea the key is to design a prgramming test which is short 30 mins to 1 hour tops. If the candidate does not bother to do it, then we will reject them since it says a lot of about their attitude towards the role. – bobo2000 Feb 03 '17 at 09:58
  • audition projects are different from coding interview questions. at least a project may involve more interesting tasks and give you the freedom to choose what and how to implement something. coding tests are usually very concentrated and frankly quite boring. –  Feb 05 '17 at 17:20
  • "good attitude towards the company and role" - imagine if the interviewee gets 10 such tasks a week and every company thinks the same. From his perspective the interviewer is out of touch with reality. – Johannesberg Apr 20 '17 at 10:03
  • @Pithikos a lot of companies these days have technical interviews. Google's is very rigorous.We are not Google no, but that doesn't mean we should lower our standards by having no technical screening process increasing the risk of hiring a bad hire. – bobo2000 Apr 20 '17 at 14:38
  • @Pithikos also just checked, Facebook have a take home interview coding assignment for some position – bobo2000 Apr 20 '17 at 14:40
  • Giving a very short (5-10min) coding excercise during the interview is fine, but you can't expect people to waste an hour of their time. The interviewees do not know that very few people complete it. Bottom line is you can think what you want, but unless they see something extraordinary about your company they're not going to go out of their to please you, not just on a slim chance that you'll call them back among the potentially tens or hundreds of people who completed the same test. They just move on to the next company. – Demonblack Jun 27 '17 at 17:55
  • Also, I just noticed you give them the test on the phone! Put yourself in their shoes for a moment. They haven't even met you. They know nothing about your company, and you're already wasting their time with what they see as pointless games. How would you react in that situation? You mention Google and Facebook, and I'm very sorry to break it to you but you're neither Google nor Facebook. Extraordinary companies can afford to require extraordinary effort from their applicants, ordinary companies can't. There's nothing you can do except deal with it. – Demonblack Jun 27 '17 at 17:55
  • @Demonblack yes absolutely, doing this means that we are going to get less candidates, but that is why when we do end up recruiting someone we have reduced the likelihood of a bad hire and generally hire people who lack experience but are extremely motivated to learn. It just takes longer to do it, and for those that are confident in their ability, it does not bother them the slightest. – bobo2000 Jun 28 '17 at 10:15
  • @bobo2000 Then I think I'm missing something here.Btw I think with your method you also have a reduced likelihood of a good hire as well, but that's beside the point. If you are actually getting what you wanted what's the issue? – Demonblack Jun 28 '17 at 10:15
  • At the time when I posted this question (a year ago) we were having problems using this process, and my question was whether it was too heavy of an approach, but have since optimised it and it works fine now. – bobo2000 Jun 28 '17 at 10:39
  • 1
    Ehm quick question, how can a programmer cheat at an assignment. 80% of our job is getting code from the internet. – EpicKip May 01 '18 at 10:12

24 Answers24

291

You need a programming test. But it should be 5-10 minutes. 30 minutes tops. There is no 2 hour test that will tell you exactly how good a programmer is. You won't be able to tell if they continually write maintainable code, or if they always comment properly, or if they come up with unreadable messes of 'clever' solutions to problems, etc. In 2 hours, the only thing you'll accomplish is find out who is outright lying about knowing a language/programming.

Except you should be able to spot outright frauds in far less time than 2 hours. 5 minutes writing FizzBuzz and 2-3 quick questions about language specific features will tell you if they are absolutely worthless. And that's about the best you can do in a job interview.

Let me tell you what I would think if I received a 2 hour exam for the privilege of conducting an interview:

"Either these people think this has value which means they have no clue what they are doing, OR, they know this is a waste of time, but for some reason (probably blindly following HR's red tape), they are willing to waste the candidate's time before the candidate is even hired. Imagine what they'll think they can get away with once they are paying me.

Either way, I don't want to work there."

There's another thing that could be driving away candidates: The length of your interview process. You have people doing a phone interview. Then a take home test that they have a week to finish. Then you go over the tests. Then you set up a face interview with management. Then (I assume) you do a couple of face interviews. Then you get back to the guy who you want to hire. What does that take? 2 weeks? Longer?

In that time I've already gotten 3 offers. I accepted one and I start next week. You think I'm going to go do your 2 hour test?

Community
  • 1
Shane
  • 2,265
  • 1
  • 10
  • 6
  • Comments are not for extended discussion; this conversation has been moved to chat. – Jane S Feb 03 '16 at 10:00
  • 23
    But it should be 5-10 minutes. 30 minutes tops. - It does not have to be this short, but having them do it on-site before they leave would increase the completion rate. It doesn't matter if its a 2 hour challenge if you have them do it in house they wont get home and decide its not worth the effort.

    But in general I agree a quick fizz buzz type challenge is likely to yield similar results to a more in-depth test with less effort required to evaluate the results. And if it takes 3-4 weeks to get through a hiring process I have already started somewhere else.

     – IDrinkandIKnowThings Feb 03 '16 at 21:37
  • I don't mind longer interview processes - I think that varies a bit. It's a question of whether candidates want a job anywhere or with your company. Give me a week or two to do my homework and figure out if/where I want to relocate, and how much that'll cost, and I'm more likely to accept the offer. Rush me, and I might not. – Sobrique Feb 04 '16 at 09:44
  • 35
    No, you don't need any programming tests. What you need is trial period. – Agent_L Feb 04 '16 at 10:52
  • 83
    @Agent_L An employee making $40,000 costs (on average) $16,000 to replace. An employee who makes $80,000 costs $120,000 to replace. That's an insane amount to pay for a trial period to weed out half your candidates when you could give a 5 min test for free instead. Seriously, why on earth would you hire someone and start paying them for several weeks/months just to find out they lied on their resume? Why do that when you can spend 5 min and not pay them a penny to find out the same thing? – Shane Feb 04 '16 at 22:00
  • but you need to know how familiar they are with the platform you'll have them working on. depending on the level of experience you require, the amount of conception and architecture that your test may require can vary greatly. – njzk2 Feb 04 '16 at 23:35
  • 2
    @Shane WHAT? You just pay them for how much they worked. If they lied, it's obvious in first few days and then you let them go (not even mentioning that if he managed to fool you in the interview, then hire him as a salesman). $40k/year is merely $158 a day. What you cited is costs of hiring wrong person based on nothing but test-solving capabilities. And taking months to realize he's not a team player. Your solution is the one costing $16k, not mine. – Agent_L Feb 05 '16 at 17:55
  • 22
    @Agent_L: Costs of a failed hire are far higher than the wages mis-spent. They include the need to re-do the hire, for example, and all the admin associated with adding a person to your staff. I do not know where Shane's figures came from, but they don't seem unreasonable. Probably that 16,000 is on top of the salary given to the bad hire. As you are proposing alternative solutions, presumably you have been a hiring manager, or work in some capacity related to hiring? – Neil Slater Feb 05 '16 at 19:15
  • 6
    it seems to me that a trial period to get a job would attract a lot of mediocre / terrible programmers while the very good ones won't come work for you for 1-2 months to find out after if they're hired. Just my opinion – Ant Feb 06 '16 at 14:06
  • "It does not have to be this short, but having them do it on-site before they leave would increase the completion rate. ..." - Personally, I would have them do it before they are invited onsite. Make them do it when they submit a resume. Why waste time with candidates that don't meet minimum or basic standards? –  Feb 07 '16 at 06:05
  • 1
    @Agent_L it depends on where you work. In some countries when you start a job you cannot just for the guy after 2 days, it may take weeks or months and in the mean time you have to pay them. – Bakuriu Feb 07 '16 at 13:32
  • @Agent_L I live in one of those countries where it's really difficult to fire someone after hiring them. You have to go through a formal process that takes months. – Gustav Bertram Feb 08 '16 at 07:55
  • 48
    It's worth adding that these same principles applies to all tests, not just programming tests. All "Homework" tests discriminate against busy and in-demand people, and tend to test the wrong things (domain knowledge that can be learned on the job, not aptitudes that can't). I had the exact same thing in a team who recruited graphic designers using a multi-hour "design X in our brand style" homework task, which I replaced with a kind of in-interview fizz-buzz for creative thinking. – user56reinstatemonica8 Feb 08 '16 at 09:58
  • @GustavBertram Yeah, I live in Poland where it's so difficult to fire someone that no one gets a real job contract anymore. Predefined-length contract, IP rights transfer, contract to perform a specified task, casual work contract - the options are endless. All you need is mutual will to work around limitations. – Agent_L Feb 08 '16 at 12:07
  • @Shane, the cost of a bad higher statistic is very interesting. Do you have a source for that? I believe you and agree that bad hires cost a lot but the numbers struck me as high. Thanks! – chessofnerd Sep 30 '16 at 04:53
  • I don't understand all the people saying @Agent_L is wrong and it will be more expensive. Trial is made so they can fire you for whatever reason and they only had to pay you a month's salary. You could say the hiring process costs money etc. but not hiring anybody is also costing them money. – EpicKip Jun 28 '17 at 14:18
  • 1
    @chessofnerd https://www.standardforsuccess.com/wp-content/uploads/2015/08/Employee_Turnover_Infographic_8.27_2.pdf – Shane Jun 28 '17 at 14:29
  • 1
    @EpicKip So you hire somebody to do a job. Let's say their monthly salary is $1000. Instead of being productive and producing $2000 profit for the company they do nothing. Then it takes a month to find someone new. With a good hire, the company could have $4000 dollars in its pocket afer sales and salary. Instead, it is $1000 in debt from spending the salary. That's $5000 the company lost. There's other factors, like how you get more efficient on the job the longer you're on it, but that's is largely the gist of it. – Shane Jun 28 '17 at 14:38
  • @Shane You cant find everything from a test. I am a programmer and any place that hires (at least in the Netherlands) will hire you based on 1 month trial (they do have an interview and maybe a very small test). And not hiring anybody because of a long/wrong hiring process will cost the company more money overtime. – EpicKip Jun 28 '17 at 14:40
  • "they do have an interview and maybe a very small test" Uhh, yeah. That's exactly the part we are talking about here. The question is about that interview portion and whether or not to maybe have a very small test. And my first paragraph goes on at length about how "You can't find everything from a test". Being from the Netherlands, maybe there is a bit of a language barrier? But, I'd suggest you re-read the question and my answer because I think we agree more than we disagreeing here. – Shane Jun 28 '17 at 14:51
  • 1
    @Shane, A coding test also doesn’t identify the most important aspect of a programmer: whether or not they’re capable of learning. – Draco18s no longer trusts SE Jun 28 '17 at 20:40
  • @Draco18s First paragraph: "One thing I’ve noticed in hunting for a job recently is the number of companies that insist that you write them a code sample to spec. Not just any code sample, but a fully functional, complete application. This is absurd, for several reasons." He goes on to say: "Joel Spolsky talks about how it’s important to ask developers to write code during their interview. But he also goes into detail about how he interprets this in his own company: namely, he asks people to write functions during the interview." This is identical to what I'm saying. 1/2 – Shane Jun 29 '17 at 14:46
  • @Shane Quick, on a whiteboard, write me a function that will determine whether or not a singly linked list contains a loop. Go. Also, you can't modify the data structure or create a new list. – Draco18s no longer trusts SE Jun 29 '17 at 14:48
  • 2/2 He finishes by saying : "Ask a few technical questions that require code in an interview." The complaint he's making about coding tests is about the kind of test the OP was asking about. This answer recommends the same thing. Also, I disagree that it shows they can't learn. It shows they learned at least something. Their coding style isn't 'google for code samples and hope'. Besides showing the ability to have learned programming 101, not much you'll do in the interview process can show someone learning over time. Or at least, I'd love to hear how you can suss that out in a few minutes. – Shane Jun 29 '17 at 14:56
  • 2
    Yes, that would be a decent kind of question -- assuming your shop works with technologies that use linked lists. Although, I'm not personally a fan of artificial limits like you have there. – Shane Jun 29 '17 at 14:58
  • 2
    A longish take home test before the first face-to-face interview is a joke. – pmf Nov 06 '17 at 09:48
  • "But it should be 5-10 minutes" - then you are going to loose many good candidates which just need some time to consider different solutions, etc. 30 min makes sense. – TT_ stands with Russia Oct 26 '21 at 02:26
202

Let me begin by saying hiring employees in general is a pathetically inexact science, and you will get various results no matter what you try. Having said that, I felt I should share my thoughts on this.

I think programming exercises are a bust, mainly because you will get people who are desperate and have way too much time on their hands completing it. If they have a full time job, you'd have to ask how much time/mental energy they are taking from their current employer to work for someone who isn't even paying them. Do you want them doing things like that to you?

In addition, if they are good, they are most likely interviewing with many employers. Guess which ones they will prioritize? The ones who don't ask to complete projects before they even get on the phone with an engineering manager.

There is also the problem of plagiarism. Sure, they will be found out in the in-person interview, but by then you've already wasted the time of (presumably) highly-paid people at the company interviewing this person.

My current company did it well, and this is the route I would go down in your position. Give them a small task which should only take maybe 90-120 minutes to do, and tell them to justify in the comments why they chose that way of solving the problem. This is something that can be done in one night, and will give you insight to their thinking.

I can tell you right now that if I get a project that takes 8+ hours to do, I tell them thank you but I'm not interested. I have a good job and a life. If a manager sees that as an issue, that tells me they wouldn't care about my work/life balance on the job (especially if they don't care before I have it). No company except maybe Google, Apple, or Facebook could get away with that.

Lawrence Aiello
  • 11,882
  • 7
  • 37
  • 55
  • Comments are not for extended discussion; this conversation has been moved to chat. – Jane S Feb 03 '16 at 10:05
  • @Aron it does not make economical sense to pay them to do the test given the amount of developers we interview. At the same time, I am open to suggestions on how to make the process better. My only concern is if I did not do this, and we end up making a bad hire because it turns out the guy cant program (it has happened), it will cost us a lot more in the long run. – bobo2000 Feb 03 '16 at 10:06
  • 15
    Disagree with others here and +1, wish I could plus more, because of "I think programming exercises are a bust". You kinda go back on it by suggesting a programming challenge, but still. I agree with you, they are ridiculous and don't work at all because I've worked beside many people who apparently passed these tests but are incompetent. In fact many people I've worked beside didn't do their actual work, but sat there studying interview type questions so they could apply to other programming jobs on the job. This process needs to die. –  Feb 03 '16 at 15:52
  • 15
    Google and Facebook both have highly optimized recruiting procedures, and neither uses at-home tests as part of said procedures. – stannius Feb 03 '16 at 19:00
  • 11
    This must have been written before the asker clarified the length of the test, because it's a terrible answer now. – DCShannon Feb 03 '16 at 21:24
  • 9
    @stannius That is not true. Facebook does have an at home project as part of their interview process. I have taken it – Kyeotic Feb 04 '16 at 04:56
  • 13
    "give them a small task which should only take maybe 90-120 minutes" This seems worse by your own description than a programming task that takes 30 - 60 minutes, which is what they're currently tasked with. – TylerH Feb 04 '16 at 07:05
  • 2
    Google also has an at-home (online and timed) programming assessment. Though it doesn't take long long at all. – jaredready Feb 04 '16 at 15:17
  • Well, either it's a new process (in the last year), it's not used for their Seattle area offices, or they don't make everyone do it. – stannius Feb 04 '16 at 17:12
  • 1
    @stannius - You're correct about the take-home test part, for Google at least, in my experience. However they will make you write code during the interview. Repeatedly. Typically on a whiteboard, and in front of several different interviewers (who are all members of the development team). Besides, the answer doesn't say "take home exercises are a bust", it says "programming exercises are a bust". Which is very different. Both Google and Facebook absolutely use programming exercises when they interview programmers. That part of this answer is just plain wrong. – aroth Feb 05 '16 at 00:20
  • 1
    @aroth I was referring to take-home programming exercises in that context, not ones that are done on a whiteboard during the interview. I would expect that for any company I interview for no matter what. – Lawrence Aiello Feb 05 '16 at 04:08
  • 3
    Remember those people in high school who were just not smart at all? Yep, those people will work with you in some form or another. They can be programmers, doctors, lawyers, plumbers, professors, you name it - there's dumb people doing that job. Problem is there are too many jobs to fill and not enough dedicated, hardworking, smart people to fill them. Therefore you get the liars, the cheats, the brown-nosers in every field in every type of job. I've heard horror stories of programmers who can't even turn on a computer, literally. No way to fight it. – noobsmcgoobs Feb 07 '16 at 11:02
  • 1
    90-120 minutes? If you told me you want me to spend two hours doing an excercise as part of the first interview you better hope your advertised salary figures are remarkably higher than the competition's, because otherwise I'm leaving and so is everyone else who isn't unemployed. Only desperate people accept this kind of waste of time, and, at the cost of being unfair, those tend not to be the best programmers. – Demonblack Jun 27 '17 at 18:02
  • So you wouldn't hire someone who works full-time and also on other personal or open source projects? You're not going to get anyone to spend 100% of their programming bandwidth for your company. I hope they could help their kids with homework. –  Jun 27 '17 at 22:28
  • 1
    Even Google, Apple, or Facebook don't ask for 8+h take-home project. Yet plenty of candidates are willing to do. Simply because take-home projects are huge red flag. – Raychenon Sep 16 '20 at 13:08
114

In my experience, take-home projects don't weed out bad coders, and likely make good coders find a job where they don't have to jump through a hoop to talk to the hiring manager.

Think about it this way. A good coder can easily get phone and in-person interviews. Every extra hoop they have to jump through will mean they'll take the easier route and interview (and get hired) by someone else who will pay just as much. If you make people jump through hoops, make sure they see it's worth their while from the get-go.

A bad developer can take as long as they want. You won't see them take 4x as long; you'll just see the completed test. You also won't see them go to Google or their buddy for help with an if statement.

My last company did this, and the vast majority of people we brought in for an in-person interview failed Fizz buzz (around 65%). I think this happened because we inadvertently weeded out good, busy people that didn't need another company's interview, and at the same time, dangling an easy interview in front of bad coders.

What I think we should have done instead

Instead of giving people a take-home assignment that took 15-20 minutes to grade, we should have done a 15-20 minute Skype interview where we asked questions like Fizz buzz. This would have taken the same amount of time, but I would likely have weeded out the bad programmers before a two-hour in-person interview.

FYI -> Here is a very detailed article about Fizz-buzz and general interviewing practices.

sevensevens
  • 24,003
  • 8
  • 56
  • 80
  • 4
    I was once asked to complete a programming task on hackerearth.com and was asked to enter personal info to register myself. It was an immediate turn-off. Moreover, the programming task itself was not a reliable indicator of capability. I agree that such take-home assignments are not really worth it. – Nav Feb 03 '16 at 23:53
  • 3
    The one and only time I've ever done a take home programming task, the hiring manager hounded me because I didn't have 1 day turn-around. After that I decided if I got a take-home assignment again, to say I wasn't interested in the job. – sevensevens Feb 04 '16 at 16:09
  • time only is not sufficient to properly complete a task. – njzk2 Feb 04 '16 at 23:36
  • Problem with the Fizz buzz interview coding practice is you have numbers that are both divisible by 3 and 5. That's the first thing that comes up in my mind, and how would you want the order in those situations. Unless you just want data. – Chad Feb 08 '16 at 01:04
  • @Chad what is there to order? http://coliru.stacked-crooked.com/a/bab3aa8f4ea44fcd –  Feb 08 '16 at 04:12
  • 1
    @TechnikEmpire you have numbers that are both divisible by 3 and 5. So you get both Fizz and Buzz being outputted in one iteration. The order which one of the two comes first is dependent how you order you divisors eg.. 3 first or 5 first. Something that I saw in your code example you've done exactly the same thing. – Chad Feb 08 '16 at 12:56
  • @Chad oh okay, just didn't know what you meant. –  Feb 08 '16 at 13:19
  • @TechnikEmpire btw just for fun I did it old school. http://coliru.stacked-crooked.com/a/31dc2f5cbbc0e76b – Chad Feb 08 '16 at 13:25
  • 1
    @TechnikEmpire Really enjoyed reading your code, interesting to see how far C++ has come. – Chad Feb 08 '16 at 13:27
  • 1
    @Chad Nice. that's alien to me because I've never used printf in my life. lol –  Feb 08 '16 at 13:30
  • Please tell me more about these if statements - are those the ones that cause a velociraptor attack? – Code Jockey Feb 08 '16 at 21:06
  • 1
    @CodeJockey Yes, if statements are evil, cmon man keep up with the 10000000s of arbitrary "standards" that random people and companies make up to decide who is more elite than the next guy. Geeze you failed the test for the interview you must be a bad programmer (this is obviously sarcasm). –  Feb 09 '16 at 03:01
  • 3
    My favorite way to do FizzBuzz if I have my choice of language (no math, if statements, or technically even loops): filter fb{@($_,"Fizz","Buzz","FizzBuzz")[0+(@(2)[$_-match'[^05]$'])+(@(1)[$_-notmatch'(^([369][0369]?|[258][147]|[147][258]))$'])]}1..100|fb – Code Jockey Feb 09 '16 at 20:30
  • 7
    @CodeJockey: And I'd ask you to write a version that I can understand. – gnasher729 Jun 27 '17 at 21:40
  • @gnasher729 - that's why you either wouldn't give me a choice of language, or you would require me (if you had the time) to explain how my example works. In the case of my example up there (PowerShell), instead of math (that is, more than simple addition and multiplication) you use regular expressions; instead of if statements, you use array index auto-selection; and instead of looping, you use a range of integers piped into a filter. – Code Jockey Jun 28 '17 at 17:27
  • @gnasher729 - this is the more readable version I added to Rosetta Code, if you're interested: http://rosettacode.org/wiki/FizzBuzz#Filter.2C_Piping.2C_Regex_Matching.2C_Array_Auto-Selection – Code Jockey Jun 28 '17 at 17:36
  • 1
    You've explained my whole issue with especially difficult whiteboard programming problems. Great programmers won't bother studying it--they're already swimming in offers. Ambitious but mediocre programmers know they're at a disadvantage and will study. The end result of that kind of testing -- which is the end result of most interviewing nonsense -- is the signal gets totally lost in the noise. – Slothario Apr 30 '18 at 18:21
  • 1
    @CodeJockey You would fail both the maintainability test and the smartass test :-) Only (somewhat) joking about the latter; but the former is extremely important. "Clever" solutions get shot down at code review – Mawg says reinstate Monica Oct 25 '18 at 08:39
75

A contrasting view, from someone in the trenches on both sides of this question. I suspect this answer will attract down-votes, but it's a highly informed opinion, developed over several decades in this business. Because I like doing so, I still write code every day for a living and as a hobby in my "copious free time", but I've also been the final decision maker on hiring a couple dozen programmers, and have helped to interview and advised on the hiring of a fairly large number more.

Lawrence is right: hiring is woefully inexact. But we're getting better at it over time. More than face-to-face chats, more than trivia questions, short on-site programming challenges are an increasingly essential part of that: not only for skill, but for attitude.

Note that I said "short". We give candidates a laptop set up with a properly installed Eclipse (our chosen IDE) when they come in for the interview. Properly set up means they have access to the jdk source, but it has no internet, and they're asked to not use cell phones during the test. They get an hour to solve anywhere from zero to five problems, starting from "FizzBuzz", and going up in complexity to something on the order of a simple multi-threaded producer/multiple-consumer system.

Nobody's ever solved all five in an hour, and I don't expect anyone ever will (I can write solutions for five in comfortably under an hour, but of course I've seen the problems and solved them all previously, so I'm not a fair test).

Incidentally, we've had ostensibly experienced programmers fail to get FizzBuzz right. Let me say that again: we've had a handful of candidates fail to complete FizzBuzz correctly. Usually by not following directions, but reading the spec is an essential part of being a developer.

We have them write code while here on premises rather than giving them homework for several reasons, partly the risk of cheating, and partly to give each candidate an equal opportunity to work the problems. We also rotate problems in and out of the challenge set, and we discuss solutions with candidates afterwards.

I feel extremely strongly on this issue. I've never seen a candidate refuse a position or drop out of the interview process because of our programming test. But that may be influenced by the fact that we tell the recruiters who serve us that the process includes on-site programming. So we might just not see the candidates who think being asked to write an hour's worth of code is "unpaid work". If so, good riddance. If they think an hour's coding is more like "work" than an hour of answering trivia questions, or that their answers to test problems can produce anything useful, I don't need their attitude.

We did once have one candidate complain about the test, because he thought he was too senior to have to prove himself (that's not speculation - he said so). And he did well on the challenges, and he had all the experience we wanted. But he was a terrible employee: he also apparently thought he was too senior to take direction or learn, and ultimately we had to let him go.

One of the things I've decided over the years we've been doing this is that any company which doesn't test is less likely to get me as an employee. Like their answers to questions like what they use for source control etc., whether a company tests tells me a lot about how good they are at the business of software development.

So, what should you do? You should do what works for your organization, but my advice is to continue the tests, but do them onsite, as part of the interview. Tell them in advance that that's part of the process, and do it happen before they meet with upper management (but you should meet them first to help set them at ease). And really: skip the interview with upper management if they fail. Don't worry about popularity with candidates or posters on the internet. The cost of a bad hire is way worse than the cost of delaying until you make a good hire.

CPerkins
  • 921
  • 6
  • 12
  • 62
    I don't expect downvotes. You're perfectly in line with the other answers, arguing in favor of shorter on site tests and against long at home tests. The one thing I personally disagree with is the "no internet" clause. For a developer the internet is one of the basic tools of their trade - taking it away shows you how well they do without internet, which is irrelevant to 99% of development jobs. – Peter Feb 02 '16 at 22:13
  • 13
    @Peter It may even be better to allow Internet access and just see where they go. Do they go to StackOverflow to check something? Do they come here to complain? (Of course, such monitoring should be explicitly stated to the programmer and require their consent to continue to avoid controversy.) – JAB Feb 02 '16 at 22:23
  • 8
    Honestly your answer doesn't contradict the other answers/general sentiment at all. IMO there is a (huge) difference between "do this task on your own time to have a shot at this position" (OP current process) and "let's assess your ability and fit for the company together" (what you're suggesting). – Cat'r'pillar Feb 02 '16 at 22:54
  • 11
    At my current employer, we do something similar to this, and this is closest to the answer I would have written. One thing I would add (perhaps you could add as an option or note for OP?): We allow internet access and do the tests as "pair coding" exercise with team members that the person would be working with. The coding partner helps explain the goals, and is there to answer questions, help things along if necessary - i.e. much like how they would work together in practice. Not only do we get to assess technical skills, but also get a good sense of how the candidate works with others. – Neil Slater Feb 03 '16 at 09:14
  • @NeilSlater I am all for the on site approach, but what if the candidate is abroad doing a technical internship? I am interviewing one candidate that is exactly in that situation in Hong Kong, who is looking to return to the UK once it's over. – bobo2000 Feb 03 '16 at 12:51
  • 2
    @bobo2000: If candidates are special cases, then you need to figure out what to do. I might run the pair programming test over video conferencing for your example, and allow a little extra time for it. Maybe this is the basis for a different Workplace SE Q&A? Either way, I don't think worrying about all the possible special cases should stop you from having a thought-through standard approach, and it is pretty normal to assume one part of the interview is on-site at the place of work. – Neil Slater Feb 03 '16 at 14:16
  • 11
    Agreed with @Peter. If I were interviewing with you, the "no Internet" thing would make me think you value memorizing an API more than understanding the concepts, which would make me think twice before accepting a job an offer from your company. Otherwise, I generally agree with the answer and the last sentence is especially true. – reirab Feb 03 '16 at 16:31
  • 1
    @reirab I wasn't clear. "No internet" doesn't mean "memorizing the api". Any decent ide (we use eclipse) links you to the jdk source. I'm on windows, so "Open Declaration" is bound to the F3 key. I hit it a lot. I'd be disappointed in a candidate who didn't. I have just edited my answer to make clear that the jdk sources are available to the candidate. – CPerkins Feb 03 '16 at 17:12
  • 4
    @CPerkins stack overflow? Maybe because I'm not a java guy, but hearing that no internet access bit would be a major red flag for me. I don't care to waste my time reinventing wheels. Unless the tasks are so basic (i.e. fizzbuzz) that any programmer could solve it immediately. I would think you'd want to see me solve problems, the IDE is only one tool out of many that are necessary for that. – Jared Smith Feb 03 '16 at 23:40
  • 1
    @JaredSmith yes that's me, why? We naturally also do verbal design, walk through previous work, all of that. But this is how we do it. I'm comfortable knowing that it might cost us a few otherwise good candidates. The "no internet access" is deliberate. It's about the candidate's ability to solve problems from their own knowledge and skill, backed up by the jdk sources where necessary to remind us (me too) about the details of the api, as opposed to googling for someone else's solution or asking a friend for help. – CPerkins Feb 04 '16 at 00:11
  • 2
    +1 explicitly for "One of the things I've decided over the years we've been doing this is that any company which doesn't test is less likely to get me as an employee." A company that cares about my code ability is a lot more likely to get my vote than a company that just takes my word about the skills and likes my personality. Those are important too, but not exclusively. – StackExchange What The Heck Feb 04 '16 at 14:12
  • 1
    @Peter I agree with you, but only for the more difficult excercises. No one should need internet to solve FizzBuzz or other trivial excercises, but you don't want to lose a good programmer just because they can't remember some obscure syntax that they used once and never again, and can be found on the internet in five seconds. Googling is as much a development skill as coding itself - I know people who are absolutely terrible at it. They take hours looking for something that can be found in minutes with the right keywords. – Demonblack Jun 27 '17 at 18:18
44

Why no at home tests?

Even if we ignore the possibility of cheating, and the reverse-filter which causes good and honest candidates to avoid companies with at-home coding tests, the value of at home coding tests is limited.

If it's for a senior position, a senior developer with people skills will be able to tell within less than 10 minutes if the senior developer on the other end of the phone is any good, or is just making up stuff. We won't know how good exactly, but we'll know as much if not more than every conceivable at-home coding test tells us.

If it's for a junior position, we don't expect much in terms of technical expertise. We're looking for enthusiasm, willingness to learn, and talent - none of which are spotted from an at home coding test. There are too many things juniors are allowed to be oblivious to. If we hire them we need to train them anyway.

How to test instead?

As a filter, just give them 10 minutes to solve a FizzBuzz variation on site during the first round of interviews or, if you get swamped with good CVs and need another filter, do it over Skype before the first real round of interviews.

Once they've passed the filters, you need to know more about the coding abilities of your candidates. I recommend spending 30 minutes to at most 2 hours of either pair programming or code reviews - actual real work, rather than an exercise. Repeat 1-2 times with different partners. The advantage of pair programming and code reviews is that the developer already has sufficient knowledge to contribute.

Don't worry about the test being different for each hire - the goal of the hiring process isn't to find the person who scores best in a couple of measurable and repeatable tests. The goal is to hire a single person who will do the job well.

Peter
  • 14,539
  • 2
  • 30
  • 52
  • 21
    You are fooling yourself if you think someone can tell within 10 minutes if the person on the other end of the phone is any good. – Dunk Feb 02 '16 at 21:46
  • 9
    Enthusiasm and willingness to learn? What about existing knowledge or ability to learn? –  Feb 02 '16 at 21:54
  • 4
    @djechlin Nice to have, but not nearly as important for a Junior. Of course if they claim to be enthusiastic about coding and willing to learn, yet know nothing, then I'd question their enthusiasm. I'm lucky enough to never have met anyone who was willing but not able to learn. – Peter Feb 02 '16 at 22:00
  • 2
    lol, so your criterion works because you adapt it to mean whatever you need when you see a candidate you like or don't. Why not just go with "has spark" or "lacks spark," it's simpler. –  Feb 02 '16 at 22:05
  • 16
    As an aside, teaching an advanced math class, I've definitely met willing but not able. –  Feb 02 '16 at 22:07
  • @djechlin You have a point. Adjusted by including "talent". – Peter Feb 03 '16 at 01:58
  • 13
    @Dunk I disagree. It would take me 10 minutes to decided if they where "ANY good". But it would take a lot longer to see if they were "good enough". The point of having interviewing "rounds" is to progressively weed out candidates. What the OP has done is create the equivalent of a "quest of worthiness". – Aron Feb 03 '16 at 02:06
  • To be honest, a lot of the juniors we've interviewed have done 1 or 2 technical internships which is why we test them. If you are a junior and have never used a MVC framework, or created a program where you have done API integration, then I am not sure if I want to bring them on board. The time and cost it takes to train someone from scratch does not make the venture worthwhile. – bobo2000 Feb 03 '16 at 12:04
  • 9
    +1000 for the "10 minutes on the phone with a senior developer" part. If you want to know whether a developer knows his stuff, hook them up with another developer whose opinion you trust and they'll sort it out faster than any test. Short, in-person tests are okay if you insist (but far from necessary) but the much more reliable method for weeding out bad candidates is by a peer actually interacting with them. – thanby Feb 03 '16 at 21:10
  • I had to scroll too far to find this! – Agent_L Feb 04 '16 at 10:59
  • 6
    Well Aron, if you could indeed weed out the people who are "any good" in 10 minutes then you are wasting your time as a developer. Companies would pay really, really big bucks to people who could feed them nothing but good developers. Heck, if you really wanted to keep doing development then you'd probably still make a fortune vetting developers one day a week and you'd have enough money to fund your own development company and do what interests you. Especially, since your company would only hire the "good" developers. – Dunk Feb 04 '16 at 21:58
  • 4
    Regarding: "Not an exercise based on real code, but actual real work", this needs some caveats/explanation. It will get you into legal trouble in many countries if your interview process involves candidates doing real work that could arguably benefit your company without them being paid. It doesn't matter if you think the amount is too trivial, or you won't use that code review - it's actual work and many legislations will require that the candidates are paid (or even considered hired, albeit temporarily, at that point). – Neil Slater Feb 07 '16 at 10:07
  • I've got 10 years exp and I got sent a contract role test which would have taken me 2-3 days to do properly, needless to say I chuckled and declined. – SuperUberDuper Mar 21 '17 at 10:54
  • 3
    You may not identify the (very) good ones in 10 minutes, but you ought to be able to identify the (very) bad – Mawg says reinstate Monica Oct 25 '18 at 12:04
31

Here is another point that I don't see covered in the existing answers (which I think cover the topic well in general). I would take a close and serious look at how long it actually takes people to complete. I have had four job interviews during which I had a programming exercise to complete, each of which was done differently (and each had its own advantages and disadvantages). Two of the four (numbers 3 and 4 below) took much longer than they said it should, and both of which I ended up giving up on because of the difficultly level involved. I've described them below and ranked them from best to worst from my perspective.

  1. During the technical interview, they sat me down at a computer that had a pared down version of their code base and had me solve three relatively short problems related to it (find and fix a bug that they had purposefully added, add a new field to an info table, etc.). They gave me exactly an hour to do it, and after an hour they went over my solution as well as how I approached the problems. This gave them more insight into me as a programmer, while respecting my time by keeping it short and to the point.
  2. During the technical interview they had me work through a problem they had encountered during development on a whiteboard while being able to bounce ideas off of them. This was the shortest option, while still giving them a chance to see how I work through problems and how much I can actually do the necessary job.
  3. During the technical interview (for a junior Ruby on Rails Web Development position) they tasked me with building a web application from scratch that would navigate to a website, fill out multiple forms as they were presented, scrape data from the resulting page, and present that to the user. They said this should be a quick exercise, and it may have been for a senior level web developer, but having had only a year of professional experience at that point, I spent four hours trying to get it all to work before giving up and going home (all of the interviewers had left hours before I did because it was an afternoon interview, they said I should just save the finished program when I was done). This is a ridiculous assignment for the listed position, gave them no idea of my coding abilities, and seemed to me like they were just trying to get free work from the deal. Needless to say I didn't even want the job by the time I left that day.
  4. Before even having a technical interview, they gave me an assignment to create a web application that would take advantage of an API their company used to "do something interesting". This was exactly as broad as it sounds. This required me to do the following before even attempting to create the application: create a developer account for the API, download the API development kit, create a publicly accessible web application (with another developer account), learn the API, and create a data repository to access with the API. This of course took many hours just to get started, and shortly after getting started on the web application itself, I got a different job interview and shortly thereafter a job offer, so I discontinued work on the assignment. This is a crazy thing to have as part of an interview process because who wants to put all of that time and effort into developing a program just to have a small chance of moving on in the interview process?

So to more directly answer your question, should you have a programming exercise? Yes, but make sure it is tailored to test what you actually care about, and not require a ton of extraneous work for the interviewee. Do you want to know about their algorithmic thinking? Give them an algorithm problem. Do you want to know about their coding style? Give them a coding problem. Do you want to know about their development process? Discuss their process with them as they work through a problem.

Kevin
  • 552
  • 6
  • 18
  • 4
    interviewers gave up and went home?! FFS, you did not want to work there anyway. I'd have left 2 minutes after they did. – gbjbaanb Feb 08 '16 at 10:01
  • They didn't so much give up as just go home from work. Once they gave me the assignment they just left me to my own devices to finish the problem, and because it was an afternoon interview and I was working on the problem for a solid four hours, they ended up going home. A couple of the employees were still there by the time I left, and I realized it was time to throw in the towel when those guys ordered dinner into the office – Kevin Feb 08 '16 at 18:23
28

Let me start off with:

  • I've been given tests to complete at home that ranged from 15 minutes to 10 hours.
  • I've been given online tests to run through.
  • I've been tasked with writing out the answer to FizzBuzz and other fashionable internet tests on whiteboards.
  • I've been asked about the square vs round manhole covers.

In short, I've dealt with pretty much all the different ways developers like to handle interviews. To be quite honest - I seriously doubt the vast majority of people interviewing me even understood what the potential outcomes of each of those tests were and ultimately just hired people on whether or not they "liked" them.

Before you even put up a job listing you should sit down and go through exactly what it is you are trying to hire and "developer" isn't an answer, at least, it's not a proper one. A good answer to this would be something like "mortgage domain expert".

Finding someone that can write a bit of code or search google for how to apply a particular pattern is trivial compared with finding someone that's been in the business you are in and can leverage that knowledge. In other words, if I was hiring for a mortgage documentation company, I'd take someone that could talk about the difference between 15 year fixed and and ARM loan over someone that could write a bubble sort algorithm in 2 minutes. The reason is that "normal" business people make all sorts of odd demands and domain experts can more easily get to the heart of what's needed whereas someone that knows nothing will happily add things that are useless or make the app really bad.

Getting back to the questions, only ask go / no go questions.

Is it critical that the person can tell you the difference between a virtual and abstract method? Might be, usually isn't. I'd wager a good portion of developers barely even know when to use those keywords but they aren't your superstars, they are the rank and file who use can't code without the visual designers.

Is it critical to spot when a query is open for sql injection? For a Sr position - absolutely; for a Jr. position no. This is something that is easily trainable and handled via code review. Hence the reason you want the sr. to know it inside and out - so they can train the juniors.

Is it critical that they know the exact maximum value of an Int32 data type? Not normally - that level of trivial knowledge is what google is for; but maybe your requirement is for coding on memory constrained devices - in that case: absolutely.

I interview and hire developers. I don't send home tests - it's too easy to have a friend help. I don't use online testing sites - given how the scoring has to be automated it's trivial to game. I don't ask candidates to write out the answer to fizzbuzz - just about everyone has seen that test and should know the answer by heart; everyone else entered the field in the last year and are jr.'s anyway. I also don't ask trivia questions - being able to recite the answer usually just means you've heard the question before.

What I do to interview people:

  • I ask them to describe a prior project or two. All I care about at this point is to make them comfortable and to get them talking. This is a go/no go because if I can't understand what they are saying then I can't work with them.

  • I ask them a few specific questions in the tech I need them to use. If it's SQL server, I'll ask about updating on a join. If it's HTML, I'll hand them a 10 line html file with a couple css classes and ask them what the output is. These are trivial questions to people with experience in those areas and quickly weeds out pretenders.

  • If I'm looking for a Sr. dev then I'll ask domain knowledge questions. Not edge case things but rather core principals. If I need you to lead a project on an accounting back end then you better have a grasp of basic accounting principals.

  • If I'm looking for a Jr. dev then I'll ask about pet projects. all I want to know is the type of tech used. This should clue you into what they really want to do. In other words a C# developer isn't likely to be doing pet projects in php. I honestly don't expect too much from a jr dev except to be trainable. If they are actively working on a pet project then I can train them. If they are the type that needs people to tell them what to do there are far larger companies these guys can work at.

I don't make these questions up on the fly, the potential answers are considered ahead of time and fit a go/no go pattern. If a question doesn't fit that then it's not relevant. I also ask every candidate the same ones unless I'm 100% convinced I'm not hiring them at which point I'll stop the interview.

If for some reason you still insist on sending home a test - make sure that the skills required to successfully complete that test in a reasonable amount of time are actually skills you care about.

I had one company give me a take home test that involved writing a custom cryptographic service provider. I completed the test because it was somewhat interesting; they hired me. At no point in my time there did I do anything even remotely related to security, cryptography or even math based beyond adding a few numbers. I wonder how many people they drove off with that test?

NotMe
  • 23,390
  • 5
  • 70
  • 104
  • Thanks for your response, good answer. We have a similar approach minus the take home test. – bobo2000 Feb 08 '16 at 15:33
  • 3
    A good developer can pick up domain knowledge better than a good domain expert can learn to program well. You're unnecessarily narrowing your hiring pool, and your company will suffer for it. – David Thornley Oct 24 '18 at 15:05
  • 3
    @DavidThornley: You and I likely have different definitions for a great, good, and a mediocre developer. In my world a great developer will ask the right questions and pick up domain knowledge quickly. A good developer may generally execute well on what they are told but doesn't pose intelligent questions about it. A mediocre one doesn't question at all and often has a redo their work more than normal. Unfortunately finding a great developer is far more difficult than finding good ones that already know the domain. – NotMe Oct 24 '18 at 20:31
  • 1
    "the exact maximum value of an Int32 data type" ... ermmm, it's INT_MAX, isn't it? – Mawg says reinstate Monica Oct 25 '18 at 12:21
18

Coming from the standpoint of a person seeking a job, I generally avoid places that take over 1 hour to code. One time I went to this place that required a nearly 3 days worth java coding project. I did it all and the guy was even impressed with it but they told me they hired someone else after the second interview stage. So after that, if I come to a company I would ignore/pass any project that requires more than a couple of hours to complete. My time is just as valuable as theirs and I rather not waste it on things that won't get me anywhere.

With that said, if your coding exercise is too long, maybe people aren't willing to put the time in. I would attempt to reduce it such that it takes an hour at most but at the same time demonstrates an understanding of coding and maybe putting a deadline on it like, "Please complete by COB tomorrow" or something. They can still "copy-and-paste" something online but make it hard by being specific rather than something you read online.

Dan
  • 4,932
  • 10
  • 13
  • A strong resume, references and being able to perform on a short test, may be all you need. Stick to your guns and good luck with the job search. –  Feb 02 '16 at 18:28
  • 12
    Amazon reached out to me a few years ago, asked me to take a 3-hour test with 3 different puzzles to figure out AND code. I stubbed out all 3 but only finished one to my satisfaction. It was kinda fun but the lack of any kind of feedback was very disappointing. A year later I had another coding interview with them but this time a member of the team came up with a puzzle and we coded it together. I didn't get the job but I learned as much in that 90 minutes of paired programming as I did in half a semester of traditional schooling. – HireThisMarine Feb 02 '16 at 18:33
  • 10
    The only time I had a "take-home" programming problem, the company never responded after I submitted my solution. I touched base with them after a week and the manager said he got it, but hadn't had time to look it over yet. They never called back, and neither did I. – TMN Feb 02 '16 at 18:43
  • "the guy was even impressed with it but they told me they hired someone else" But not impressed enough to hire you? Did they tell you why? – Nolo Problemo Feb 02 '16 at 21:19
  • @DaveisNotThatGuy It's like the OP's test. The test was given just to get a face-to-face interview; it's not used to hire them but rather just to make sure they're knowledgeable before they consider them. They were impressed I completed the assignment and with correct results. I guess they just weren't impressed with my interview. – Dan Feb 03 '16 at 19:45
  • 2
    So even though they later found you to be a fine coder, they knew before you left the initial interview that you were in the no-hire column. How could you have done the coding test differently to make up for that and advance to the next round? Hardly seems fair to you. – Nolo Problemo Feb 03 '16 at 23:51
17

I used to be a firm believer in coding tests and whiteboard coding, but I've started to realize that they're pretty much useless, because

What are you testing, anyway?

A whiteboard test, or short programming test gives you some insight into the individual, but really not that much. Unless you plan on having someone spend their time writing code on a whiteboard or FizzBuzz style code.

What do you want?

You want someone who is:

  • passionate
  • willing to learn
  • able to solve problems*
  • resonably technical
  • going to help your team improve

*Note, most developers solve their problems by knowing what terms to search for in Google.

The last thing that you want to hire is someone who is not a good fit for your team because they will drag it down.

How Can You Test For These?

  • Ask them about a project that they enjoyed. If they seem reluctant to talk about it, try expressing gentle disbelief, e.g. "You can't do X, can you?" If it's something they're passionate about, they will correct you. This will also help you learn if they're technical or not.
  • Ask them about things that they've learned recently. Or what they learned from the project they worked enjoyed.
  • Ask them about the last time (or a time) where they were missing some knowledge. How did they get the information they needed? Did they go to a teammate? Did they search the internet?
  • Ask them if there's anything they would want to see improved on their current/last team. Did they need better commit messages? More code reviews? More testing? More automation?
  • Ask them what technology excites them, and why.

If you have someone who is comptent technically in on this conversation it will be the easiest thing in the world to tell if this person will be the worst. An example: we had a kid come in and interview - he said that on a scale of 1-10, his Java skills were like a 7-8. I don't think he even knew that Java was compiled to a jar file which was later compiled to machine language by the JVM. I'd rank myself maybe a 2 or a 3 and I know that.

He was actually asked the same question by our CTO in an interview the previous day, and our CTO called him on it and explained that there was no way he was a 7-8.

Our CTO also asked him about his favorite project, which had to do with hand-held scanners. But he couldn't explain anything about how they worked, or the fact that they used polling to prevent busy-waits. He couldn't explain a single technical thing.

That guy didn't get hired.

Figure out the kind of attributes you're looking for, and then figure out how to test for those

But I really need to see how she codes!

Okay, that's fine - but unless she's going to be coding in a silo, the best thing to do will be hire her as a contractor for a small, well-defined project. Maybe you're adding the capability to download some files from an FTP and then dump them in your database. Something simple, that doesn't require much/any domain knowledge.

Have your candidates work with an existing employee or two, and pay them for their time. You'll get to see exactly how they work, and how well they work with the team. Do they communicate well? Do they get frustrated easily? Are they persistent?

There are things that you can do to hire better employees... but a programming competition probably isn't one of them.

Wayne Werner
  • 745
  • 5
  • 10
  • 8
    You raise some good points, but a drawback of "the best thing to do will be hire her as a contractor for a small, well-defined project" is that most candidates will prefer a full time offer rather than contract-to-hire. If they have multiple offers, they will take the more "secure" option. – RQDQ Feb 03 '16 at 12:28
  • @RQDQ agreed. I think it's the least effective means of interviewing... but it's the most effective form of interviewing with code. – Wayne Werner Feb 03 '16 at 12:49
  • 2
    The old "try before you buy". Though the whole industry is going that way, and my next position will likely be a 3-6 month contract to hire, I am saddened by the fear enshrined in it. When did we stop doing our homework and when did we become afraid to fire anyone? – Joshua Drake Feb 03 '16 at 17:23
  • 2
    @JoshuaDrake unemployment, that's when ;) – Wayne Werner Feb 03 '16 at 19:31
13

As others have noted, some developers may be put off by a 1-2 hour programming exercise to apply for a job. What may work better is having a white-boarding session, where the candidate solves a problem on a white board during the interview. This allows you the opportunity to have an in-person interview while also making sure they have technical chops.

These problems should not be extremely difficult, or designed to trip up a developer. A classic example is FizzBuzz. This allows you to see how they think, and also know that they can at least program and don't need to google how to do a loop.

disappointed in SO leadership
  • 814
  • 1
  • 11
  • 18
  • 5
    Does anyone really fail FizzBuzz? – Patricia Shanahan Feb 02 '16 at 19:23
  • 15
    @PatriciaShanahan: Yes, all the time, you'd be surprised at the range of quality. A side effect of developers being in demand, is many people want to take a shot at being hired as one, even if they are not quite ready or good enough, they may convince themselves that they could learn on the job. A decent short technical test really does help prevent accidentally hiring someone who can talk the talk but not deliver in practice, and that situation is common enough that you need some kind of interview technique to filter them. – Neil Slater Feb 02 '16 at 19:33
  • 2
    FizzBuzz usefulness discussed in Programmers SE: http://programmers.stackexchange.com/questions/15623/fizzbuzz-really – Neil Slater Feb 02 '16 at 19:43
  • This has worked really well for us. During a good interview, the whole team ends up around the white board over engineering the solution with the candidate. =) – ThatGuy Feb 03 '16 at 00:18
  • 7
    @NeilSlater - FizzBuzz was great when it first came out. It was still a good discriminator five years ago when that discussion took place. It's not so great now. One thing that the US education has excelled in is teaching kids how to overachieve on a standardized test. FizzBuzz (along with anything like it) has now fallen to the status of a standardized test. – David Hammen Feb 03 '16 at 04:27
  • 5
    @DavidHammen: Interesting. However, you could just redefine "FizzBuzz test" to mean "any simple coding challenge similar to FizzBuzz". Coming up with a similar problem should not be hard. – sleske Feb 03 '16 at 14:03
  • @PatriciaShanahan I've done 2 interviews in my current position. FizzBuzz success is 0/2. – disappointed in SO leadership Feb 03 '16 at 14:25
  • @NeilSlater A lot of people are aware of FizzBuzz now as well. Note I said "A classic example is..." I did not intend to suggest that FizzBuzz is the end-all-be-all of a whiteboarding test, however, the general idea remains the same. Give the candidate a trivial problem to solve, and watch how they solve it. – disappointed in SO leadership Feb 03 '16 at 14:27
  • 1
    @NeilSlater I would agree IF we did not still see candidates fail FizzBuzz. That said, we do not do FizzBuzz in isolation, if they simply write a basic implementation without mistakes or discussion, we ask them to implement it in a different way. The follow up tends to show those that understand what is actually happening versus having memorized a given implementation. The follow up also checks some human interaction skills. – Joshua Drake Feb 03 '16 at 17:10
  • @PatriciaShanahan: I'm quite sure they don't fail in understanding FizzBuzz, but understanding the requirements. For example printing both the number and the text (eg. 1, 2, 3Fizz, 4, ...), when the requirements ask to print just the text (eg. 1, 2, Fizz, 4). Or if they are more stringent (hehe), it's not enough to append separate Fizz and Buzz to create FizzBuzz, but instead you should use a third string "FizzBuzz" on modulo 15, for example. – Juha Untinen Feb 04 '16 at 10:44
  • 1
    FizzBuzz, if I was asked it in an interview I'd use it as an opportunity to weed out bad interviewers. It tests one thing "do you know what the modulo operator is". Something I rarely use in production code. What a complete waste of time. – Adrian Thompson Phillips Feb 04 '16 at 10:52
  • @AdrianThompsonPhillips That was my initial impression, until I saw people with 10 years of experience not knowing the difference between an assignment operator and the equality operator, or not knowing how to write a loop, etc. There are, unfortunately, some very under-qualified and underwhelming candidates. – disappointed in SO leadership Feb 04 '16 at 14:33
  • 1
    @silencedmessage I totally agree, asking questions about those things would be much better than obscure historical questions about linked list, modulo, shifting bits and swapping values between 2 registers. – Adrian Thompson Phillips Feb 04 '16 at 16:20
10

DO NOT GET RID OF THE CODING TEST!!!!

If you want to be a programmer you have to write code that does something. Answering trivia questions is not that important. Chatting about past experiences is good, but it is in no way a substitute for being able to code.

Do what it takes to attract people to want to take the test and get this job. Make it shorter or pay them. Bring them in on a weekend so everyone has time to observe this person in action.

This is really contingent on having people who can interpret the results of the tests/observations. You may want to hire a third party to do this screening if you don't know how to do it. Just getting the answer right, is only half the battle. Developers need Googling skills, but there is much more to it. If one programmer can do the test faster, that is a plus.

Depending on your project, you may prefer fluency in a language, so the dev can start working efficiently right away. On the other hand, you could suffer in the long run when you opted for the less skilled programmer who is going to get stuck with a key feature of your app or they build such a poor structure, it's difficult to alter and maintain. You may have been better off with a good programmer who is willing to adapt and learn the language needed. There are many devs who will absolutely run away from certain stacks, frameworks, languages, tools and operating systems.

Developers are critical positions that are very expensive to fill. I understand your concern over chasing too many good ones away, but you only need one. Don't skimp.

  • 1
    Your job is to make sure you don't hire a bad programmer and not to properly evaluate everyone who applies. Mistakes will be made. Hedge your bet and don't make the worst one. –  Feb 02 '16 at 16:12
  • 3
    That is exactly the reason why we have introduced it, there are too many cowboy developers out there who have a zillion languages on their CV but when it comes down to it cannot code. It is also the highest paid position in the company, so we can't afford to take the risk with a bad hire...it has on the flip side saved our ass a few times, a couple of guys who I suspected were weak candidates on the phone following a technical interview, withdrew their job app at the last minute once I introduced the coding test in the second round. It has crossed my mind to just do the tech phone interview. – bobo2000 Feb 02 '16 at 17:32
  • 2
    @bobo2000 - you really have to weigh the length of the test against the availability of talent in your area. Unfortunately, if it is a programmer's market, you may have to make concessions. As a sort of "A/B" test, you could randomly pick a few candidates to give a shorter version of the test and see if they show more interest. –  Feb 02 '16 at 18:24
  • 12
    @bobo2000 There's something I don't understand: in other comments you say that you have mostly fresh graduates as applicants, but here you say that you're looking to fill the highest paid position in the company. Something does not really add up. Can you explain how these 2 facts fit together ? – Radu Murzea Feb 02 '16 at 19:26
  • 4
    A coding test is one thing. A coding test that takes 8+ hours to do is another thing completely. – NotVonKaiser Feb 02 '16 at 19:47
  • 23
    @bobo2000 "I give some candidates (mostly fresh grads) the programming task" + "It is also the highest paid position in the company" = something isn't adding up here. – Esoteric Screen Name Feb 02 '16 at 20:37
  • 7
    The sense I'm getting is that the "fresh" grads are no longer falling for the promise of romance & riches from the thinly financed startup. Good for them! I did that twice last bubble, and neither of those dumps are still around. – Nolo Problemo Feb 02 '16 at 21:41
  • 1
    @RaduMurzea: In the start-up world, coders, even fresh-out-of-college coders tend to be the highest paid people around followed by the sales guys then the CEO/founders. After a couple of years you'll typically find management salaries increase as they no longer consider it too risky to spend a little more lavishly rather than re-invest. After around five years you'll find sales people earning more than programmers if the company is making good money. But in the beginning, the founders typically spend a lot of money ensuring they get the best programmers they can afford. – slebetman Feb 03 '16 at 03:20
  • @slebetman - as he said.For a resource strapped start up preventing hiring a bad coder is key to growing since they are building the product. – bobo2000 Feb 03 '16 at 10:55
  • 1
    if you're relying on fresh grads to build your business, you have a problem. – sevenseacat Feb 04 '16 at 02:02
  • 1
    @sevenseacat to be honest, most small companies cannot afford a senior dev. A good senior dev is going to cost in the region of 50-100k, and a midlevel is in the 40-50 range. If the start up is in a period of growth, which ours is, then you are limited to juniors. The trick is to find one who has already learnt a lot doing internships. They will probably move on after a year or two (after getting the start up experience), but by then there will hopefully be more resources. I also take training very seriously here. – bobo2000 Feb 04 '16 at 10:24
  • 1
    @DaveisNotThatGuy I have worked in many as an ex developer, and I agree with you. The problem with many is that they take a junior on, and do not invest into their training, rather, they expect them to code like a senior. Now that I am management, I have the chance to do differently, providing that they have a good base to work from. – bobo2000 Feb 04 '16 at 10:30
  • 4
    Ask yourself are you really going to be able to train a junior developer? Are you qualified to train a junior? Do you have time? Do you have other, experienced developers to do this? What's your training plan? What are your expectations for progress? Do you have published standards for building your code by which you can measure the new guy? Most startups don't have any of these. If you can't answer any one of these questions, don't hire a newby. You and he will regret it. – Nolo Problemo Feb 04 '16 at 18:29
  • 4
    this is why I avoid startups like the plague, that makes no sense. "Hire someone who will likely not do a good job, but they're cheap, so you have to spend much more later cleaning up after them." – sevenseacat Feb 05 '16 at 03:49
  • Don't call me in on a weekend to interview. I reserve my weekends for other things than work (barring crunch time, of course). If it's worth giving me a supervised programming test, it's worth supervising it during normal hours. – David Thornley Oct 24 '18 at 15:25
10

In my opinion, the problem you have here is not necessarily the programming test. First you have the telephone technical interview and then a work from home test before a face to face interview. It sounds like you are keeping your candidates at a distance and leaving it to the last minute before meeting them. At what point do you expect candidates to decide that they want to work for you ?

I assume your recruitment advertisement is similar to most and so focuses on location, salary and a (wish) list of skills. Candidates don't really know what they would be working on, anything about the environment or people they would have to interact with. You haven't sold the job to them yet here you are asking them to prove their technical capabilities twice before they get to ask a single question about the work.

I suggest you try changing the format of the technical phone interview to be a 30-45 minute chat about the job including plenty of opportunities for questions from the candidates, then 15 minutes of technical questions as a screen so you still get a chance to pick the better applicants without making it too onerous.

I would also consider moving the programming challenge to be done onsite prior to interviews. It would seem more achieveable to the candidates, give them an incentive to stay on track with the process and you would get the benefit of observing the true time spent on the challenge (I think you might be surprised).

koan
  • 280
  • 1
  • 6
  • Thanks for this, what should I do if the candidate is not in the same country so can't come onsite to do the task? I am interviewing a candidate tomorrow who is in Hong Kong, we are in the UK. After the face to face to interview we expect to the candidates to decide whether they want to work for us. – bobo2000 Feb 03 '16 at 17:37
  • 1
    You have to adjust the interview according to circumstances. Still, you have to sell the job to the candidate and not just throw tests at them. Furthermore, the whole process is an exercise in building confidence between the two of you. You want your candidate to know exactly what they're getting themselves into; if an international move is required then you might want to spend a little more time getting to know more about each other. – koan Feb 03 '16 at 20:31
  • Use screen-sharing if someone is not able to visit in person. – user70848 Feb 05 '16 at 23:44
  • @user70848 The point of the programming challenge is to make sure the candidate can handle a programming task. Screen sharing is a horrible way to do that because you would have to spend an hour watching the candidate and they would undoubtedly feel more self conscious than if they were doing it for real. If on the other hand the job requires pair programming then this would be a good way to do it remotely. – koan Feb 07 '16 at 16:16
  • @koan That is true, but if you're located half a world away from someone, literally, it's a much cheaper and simpler solution to help both parties make a decision on whether to invest more time and energy. It's not like you can't have another programming task, if and when you again meet in person. – user70848 Feb 09 '16 at 04:35
8

I don't like take-home tests as part of interviews, for many of the reasons people have already mentioned - extends the hiring process, devalues applicant's time, may not get a call back anyway, etc.

My main issue is that it's unrealistic to how the team actually works and makes the interview process one-sided. An applicant wants if this place is a good fit for them, including the culture, how the team approaches and solves problems. You primarily are also looking for fit, including how they work and if they have the right skill set. A take home test doesn't provide an applicant an opportunity to evaluate the soft qualities of a workplace, and employers don't get to learn how the applicant approached the problem.

A better solution might be to give the applicant a more open-ended problem that can be solved in any type of creative way. You could even restrict it to X language. Rather than email it in, invite them back to present it to yourself and upper management. It gives them autonomy and incentive to do well, because it promises another interview and shows you care to know what they think.

If you must use a test to screen which candidates make it to interview with upper management, I would include the test in the interview so you can discuss their thought process together.

user70848
  • 2,678
  • 13
  • 36
8

Do you want to hire programmers who can't program?

I'm going to venture that you do not.

Hiring programmers who can't actually solve problems and write code is a good way to ruin a tech company. And you're not going to be effective at weeding out the programmers who can't actually program (and there are a lot of those out there) if your hiring process doesn't include some sort of programming test.

Are you willing to lower your standards because everyone's trying to hire programmers?

Maybe you are, but I don't think you should be. As has been noted in the comments and answers, there are candidates out there who aren't going to bother doing programming exercises as part of an interview process because they just don't need to in order to get a job.

But are those really the people you want to hire anyways? The ones who follow the path of least resistance, do whatever is most beneficial to them in the short term, and don't really care enough about your company to complete a simple programming exercise? Those don't seem like positive attributes, and they don't provide much confidence in terms of being able to retain those candidates long-term (which is also important for a tech company, as learning curves tend to be steep and the cost of replacing existing staff is very high).

So let those other businesses have the programmers who can't even be bothered. You don't want to hire them anyways. Unlike them, you have a plan. One that's not based on the "a programmer is a programmer" fallacy. Your focus should be on quality and sustainability, not bodycount.

Is scaring off candidates a problem?

Generally no, as long as they're being scared off for a good reason. You don't want to hire people who aren't up to scratch. And some of the people who say they "can't be bothered" due to high demand might actually be using that as an excuse to cover up a "not really that good at programming so would need all week to complete a 1 hour exercise" situation.

It's good to scare away those candidates. You want to hire the capable, motivated candidates. As long as you're not scaring those ones away too, you're good.

Every candidate you don't scare off is one you have to try and evaluate. And that can be hard to do if you're not giving your technical candidates any technical exercises to use for evaluation.

How can I improve our hiring process?

Check the content of your programming exercise. Is it reasonable and appropriate for an interviewing context?

You don't want something that's going to take days (or even hours) to complete. What you want is something simple to weed out the people who just can't program, ideally with enough room for nuance that the people who can program really well can differentiate themselves. Keep in mind what you're trying to accomplish (weeding out unskilled and non-serious candidates), and ensure your content is tailored to that goal. Don't go overboard.

If you've already got some technical staff, you can use them to sanity-check (and/or help design) your exercise.

And also consider how you administer the exercise. If you just give them some documentation and say "here, do this over the next week and e-mail it to me", that's probably only going to be minimally effective.

Better might be if you can run the exercise through a web portal, which candidates can check in and start the exercise, and once they start a timer starts counting down from 1 hour. Then they either submit something within that hour, or not. That's less open-ended, better retains the candidate's focus, and provides a clear deadline/timebox both so that 1) you're not left waiting around all week for a result that's never going to come, and 2) unqualified candidates don't throw away a week of their time trying to complete your programming exercise. They get 1 hour, they either solve the problem or they do not, and you know the outcome immediately.

And even better would be to bring them in for an onsite interview. Introduce them to a member of your development team. Shut them in a room together with a workstation. Have your developer start with some general/soft interview questions, and then they can pair-program with the candidate to solve the programming exercise. This will tell you not just whether or not the candidate can code, but also how well they work with your team. Your developer should also be able to glean a lot of additional information that you just won't get by looking at a bunch of code that a candidate wrote and then e-mailed to you.

Bottom line

No, you don't want to get rid of your programming exercise. But you may want to review it for appropriate content, make sure it doesn't take too long to solve, and also look at how you're fitting it into your overarching interview process.

A self-directed take-home exercise probably isn't the best approach. But the solution to that isn't to scrap the exercise entirely. Not unless you're okay with hiring crap programmers, at any rate.

Better to scare away a lot of bad candidates and a handful of good than to open up the floodgates and hire a few bad ones.

aroth
  • 9,509
  • 1
  • 37
  • 47
  • 2
    There are more jobs than people, any tech lead can spend a few mins reading someones code on github and despense with the need for silly homework assignment's on build me a simple UI app for free to consume JSON. – SuperUberDuper Dec 07 '16 at 08:22
  • 2
    You say "don't really care enough about your company to complete a simple programming exercise": most companies are entirely interchangeable -- they are neither world savers nor world destroyers. Why should anyone care about them before establishing a relationship with the company and it's employees? Don't look for care, look for a better way to identify ability. – jmoreno Feb 04 '18 at 18:21
  • 1
    You don't want to lower your standards because it's a hot market for programmers. You do want to make it easy for them to apply and avoid additional hoops to jump through because, in a hot market, the good people are going to apply to companies that make it easy to apply to. You get to hire from the people who can't get a good job elsewhere. – David Thornley Oct 24 '18 at 15:22
8

As a direct reply to Bobo's answer (which is the accepted answer because the guy wrote it and accepted it himself, which frankly I feel is a bit pathetic):

You are coming from a totally wrong premise. I don't want to work for you. Where do you get that idea from that someone wants to work for you? You are just one of many companies offering a job. I don't want to work for you. I want to evaluate your company, and if you come out on top of all the others, that's when I want to work for you.

There's a dozen companies where I can work, you are just somewhere in the queue. I first look at what companies there are, send them my CV, they can read it and be suitably impressed or not, then you usually have a quick phone conversation where they show me that they have an interesting job and I show I have the skills, and any coding test might come at the very end.

Your take home test puts you at the very very end of the queue. To compensate you'd have to post a salary range that is £10k higher than others. Finding a job is time consuming anyway; someone who makes it ten times more time consuming is at the end of the list. If I have the choice between sending a CV and having a phone interview with ten companies, and doing homework for you, guess what I'll do.

So what happens is you find candidates that want to work for you because they won't get a job elsewhere.

gnasher729
  • 169,032
  • 78
  • 316
  • 508
7

One thing no-one appears to have mentioned; even if the test is not your issue, you should be looking for other ways of attracting talent.

If you are hunting for good people based on their existing published work, you don't need to run the test on them.

If you're just being sent people via recruiters and filtering, you are vulnerable to the fact that your needs and the recruiters' needs do not perfectly synergize. They want to place someone with you. You want a top level engineer. If they can find a top level engineer that's awesome because you'll come back to them, but if they can't (and top level engineers tend to have stuff going on that leaves little time for interviewing), they'll settle for just placing a moderate engineer with a nice suit. The loss to them is a bit of long-term rep, but that's minor compared to missing their targets. If this is provably not true in your instance, hang on to that recruiter and never let them go(*), recruiters who are prioritizing a long-term relationship over targets are precious diamonds in an ocean of coaldust.

What you want to be doing is finding provably interesting candidates. Hunt through StackOverflow and GitHub for top engineers (I hear StackOverflow has a tool to help with this), check for technically interesting companies that produced good software but who screwed up their financials or monetization, and just laid off 10 top level engineers. Spend time working at universities assisting with final year projects. Identify good potential candidates and befriend them, preferably in person, alternatively via remote; even if they have offers, good engineers hang out with good engineers. Also, they can tell you how they feel about your hiring process.

Does this sound like a lot of work? It should. One of the reasons that hiring seems so "hard" is because you're trying to do it as efficiently as possible. The more time, brainpower and resources you divert to it, the easier it is. Whether those resources would be better spent on shipping product is the eternal management question. But if you're spending a lot of time on "crap programmer filtration", that's burning money. At least the steps outlined above have inherent value beyond the hiring process.

(*): Hell, hire them.

deworde
  • 2,309
  • 16
  • 20
  • Yep, that is currently my approach. You would be surprise at the amount of developers who do not have publicly available 'git repos', – bobo2000 Feb 03 '16 at 10:34
  • 4
    Good point. However, there's a risk in relying on published code - not everyone codes in their spare time (different hobbies, family obligations etc.). While many people assume that "codes in spare time" correlates with technical ability, to me it is not clear that this is true. I know good developers who do different things outside work. – sleske Feb 03 '16 at 14:11
  • @sleske that is true, I code in my spare time (CS Major/ex developer) and I never publish my code in public repositories. But that is probably part of the problem, a candidate could easily go to a job interview and lie about their skill set using buzz words. – bobo2000 Feb 03 '16 at 14:22
  • 1
    @sleske I agree, which is why "check git repos" is a small statement in a much larger paragraph (and I've actually deleted it now). If you're looking at a professional developer who doesn't publish their work, then look at the company that they worked for. Is it a good company? Does it do good work? Why did they leave it? What aspect did they work on?

    There's lots of associated data you can use that's more useful than your tests; and if they don't have that data, then you can use the tests. But if they do, use it instead of the tests.

     – deworde Feb 03 '16 at 16:37
  • @bobo2000 Published work extends beyond github samples; see my comment above. – deworde Feb 03 '16 at 16:41
  • I am always amused when people say they reached out to me because they are amazed by my github/public projects, and then they still want me to spend 3 days interviewing with them. – grasshopper Feb 06 '16 at 13:52
  • 1
    @grasshopper See, I can see it from a "what are you like to actually be around" perspective, but they shouldn't be asking you if you know what a boolean is. – deworde Feb 06 '16 at 21:04
  • @grasshopper well said good sir! Its so annoying. I feel bad when they send the homework test and go tell them to go away, my time is more valuable. – SuperUberDuper Dec 07 '16 at 08:24
  • @SuperUberDuper good lady in my case. – grasshopper Dec 07 '16 at 23:02
  • @grasshopper I'm sure a pretty one at that ;) – SuperUberDuper Dec 07 '16 at 23:23
  • Re "Hunt through Stack Overflow and GitHub for top engineers": Stack Overflow shut down their recruiting platform. – Peter Mortensen May 27 '22 at 06:38
7

I think that you have phrased your question a little bit wrong, but the way you've phrased it reflects a common misconception about hiring programmers. Are the candidates being "scared off" by the programming task, or are they filtering your company out of their own consideration because of the task?

An anecdote to demonstrate my point: while job hunting not long ago, I saw a position for a company which seemed average. The way they described their programming process made it sound quite good, but there were very few details, so I was skeptical. Maybe they were a good place to work, maybe not. But I figured I'd see about getting to a phone screen so I could figure out the details and see if they were as good as they seemed.

I clicked on the job posting, and was immediately asked to write a cover letter. Ugh. I think every candidate hates cover letters. I didn't know this company or use their products, so what could I say about them? I googled them, read through their website and product offerings, figured out where I was likely to fit into their org chart if hired, and came up with a few paragraphs "selling" myself.

Next I provided my resume and access to my LinkedIn - but immediately afterward they asked me to fill out my relevant work experience with dates and descriptions. This info is both on my LinkedIn and my resume, it was ridiculous to have to provide the same information 3 times. I closed the browser tab. 5 minutes later I was applying to another company that offered some really cool benefits which the first did not. I could actually apply to another company with better benefits faster than I could jump through the hoops the first company wanted from me.

You need to be sure that your candidates are invested in your company in particular before presenting them with any hoops to jump through, or else they won't jump. Do you do this?

Examples of quality benefits I commonly see tech companies offering:

  1. Remote working.
  2. Free computers/monitors as a signing bonus.
  3. Company contributes to respected open-source projects.
  4. Reimbursement for professional training and/or conferences.
  5. Catered lunches.
  6. Flexible hours.
  7. Opportunity to work with new or unfamiliar technologies.
  8. "Startup culture" - aka lack of politics/bureaucracy.
  9. Company Equity.
  10. Name recognition: your company or your product is well-known. Candidates like to mention where they work and hear people respond with "Oh, neat! I like their products."
  11. Charitable or revolutionary company goals/vision. People like to write code that makes people's lives better.
  12. Above-average pay. Money covers a multitude of organizational sins.
  13. Annual company retreats to cool places.

This list isn't comprehensive, but if your company doesn't offer one or more things like the items on that list, then any obstacle in your hiring process could send a candidate looking for one that does.

So let's say that, for whatever reason, you don't or can't offer incentives like the above for your candidates to convince them to spend writing code for you for free. Then what can you do? I have two alternatives most candidates will prefer over busywork programming tasks:

Alternative #1 - Pay them an hourly rate to do your programming task as though they were a contractor. This encourages them to take it seriously both for professional reasons and because... they're getting paid. This costs you money, but so does any form of recruiting. If you're really good you can even find a way for them to diagnose and fix an actual bug in your code, in which case you're getting something useful for your money.

Alternative #2 - They have probably already written code for free which they will show you if you just ask. Most programmers have code on Github, Bitbucket, Q&A websites like Stack Overflow, or could provide you with some code they have not already published.

Why make them write code they don't care about when you could let them share a passion project with you instead? It's guaranteed to be less boring than reading yet another solution to the same generic problem for the 100th time. And since the code is already written it saves both you and your candidate time. Plus you'll get a look at what sort of code they enjoy writing, which gives insight into their personality and how well they will fit with your company culture.

JessieArr
  • 231
  • 1
  • 3
  • 1
    re: above-average pay: In my hiring experience, you pay the best, you attract all kinds of freeloaders who think they can blag the role and are only interested in the money (you see this in C-level type all the time :-) ). Never attract people with just the money. – gbjbaanb Feb 08 '16 at 10:12
6

What is your business problem?

Disregarding all the arguments for or against particular tests, it all comes down to a tradeoff - more filters will scare off some good candidates, less filters will let through poor candidates that you might have to replace soon after hiring.

It all comes down to looking at your business situation instead of general practices.

Do you currently have a problem where you lack qualified candidates and are unable to hire as quickly as you need to meet your business goals? You need to ditch this initial programming task.

Do you currently have a problem where you are not satisfied with the quality of your recent hires? Then you need to implement more filters like this.

Do you have both of these problems, and both are equally painful? Congratulations, you have found the right balance for this tradeoff.

Peteris
  • 4,071
  • 21
  • 31
5

I really think it can help establish who really knows the languages listed on their resume.

If that's really your goal, I would consider a different approach. As other answers indicated, YES, you should always have a coding question, but coding questions rarely get into details of the language. Some questions I've seen that are useful for testing this:

  1. Compare and contrast Java and C (or whatever two languages are relevant, are on the candidate's resume, etc.)
  2. What features do you think should be added to the language? (Or better yet, what do you think about this particular proposed/recent change to the language?)

Engineers who've seen a thing or two know how to answer these questions pretty easily, and in only a few minutes.

James Kingsbery
  • 504
  • 2
  • 5
  • ..or write fizz buzz. –  Feb 02 '16 at 21:56
  • 4
    BTW I can pass both these questions in many languages I can't write a hello world in. –  Feb 02 '16 at 21:57
  • I've found them useful. It is extremely hard to "pass" them if you can't say something intelligent about how approaches are different between languages, how memory allocation is different, how concurrency primitives are different etc., and as great as the FizzBuzz question it doesn't lend itself well to discussing the same. – James Kingsbery Feb 02 '16 at 22:17
  • 4
    @James Agreed. And even if they can't write a hello world in Java, if they can compare and contrast Java with various other languages they are probably more useful than someone who can write a hello world, but cannot compare and contrast Java with other languages. – Peter Feb 02 '16 at 22:32
5

IMHO there is almost 0% chance that a fresh college graduate would be able to do a tough programming code at an entry level. If your coding test takes one week time to complete then you should clearly mention in your requirement that you people are looking for programmers with at least 2+ years of experience because I think that much experience should be required for completing a code work which requires one week to complete. And I think if you are looking for fresh graduates then design your test accordingly and you can find lot of idea on the internet or else you can ask senior programmers working for you to design a suitable test for fresh graduates.

Rolen Koh
  • 433
  • 3
  • 10
  • I wouldn't personally expect a senior developer to write a suitable test, unless they had some experience on what works or not during an interview. It is very easy to become focused on what you know yourself as being easy and obvious, and hard to correct for that, which will needlessly fail candidates who don't know precisely what the test writer does. I think this situation is one reason by tests like FizzBuzz became popular - they seem(ed) to work at the right level of shared knowledge to filter candidates correctly. – Neil Slater Feb 03 '16 at 10:15
  • 1
    @Neil Slater:- It could be possible that a senior programmer would not know how to design a test for an entry level programming candidate. But it is not that difficult to know what to expect from an entry level candidate. And when my previous company wanted to hire a new programmer I knew what to ask/expect from a new programmer and I did accordingly and we hired a good candidate. Also at many places where I have been interviewed for programming job I was interviewed by programming seniors of that company. So that is why I suggested that and it is a very common practice. – Rolen Koh Feb 03 '16 at 10:47
  • 2
    Actually takes 30 minutes if you know web development. I give them 1 week since I interview many candidates mid week, and it gives them time to do the test over a time frame that suits their schedule; personal commitments, work, studies etc. I would personally prefer candidates to do the test quicker because it saves me time waiting on them. I am thinking about ditching this and just doing fizzbuzz to make my life easier. – bobo2000 Feb 03 '16 at 11:56
2

Unfamiliar mouse or unexpected keyboard layout (especially Mac vs PC), or different IDE may slow performance dramatically without any difference in competence. Also, a complete application often requires lots of boilerplate code that a developer may not have enough time to type in or even not remember. Starting a new app completely from scratch is actually a very rare task; most of the work concentrates on extending or improving the existing code.

Hence I suggest to give only very short and more carefully prepared tasks during interview. Best is to ask writing a function that must take given parameters and return the explained result and I would advice to do on paper, avoiding computer at all.

eee
  • 4,784
  • 4
  • 27
  • 44
2

Thought that I will answer this question, it has been a year since it has been posted, and we have stuck to it.

FINDINGS

Positives of approach

1) Take home test weeds out the demotivated candidates, and replaces them with candidates that really want to work for you. This alone makes it a worth while thing to do since motivated people = productive employees. If they cannot be bothered to do a 1 hour assignment then that says a lot about their attitude to getting the job.

2) I agree with others, that the take home test should not be longer than an hour - mine is very simple. I have had the following results from adding it into the recruitment process-

a) Some candidates do not complete it. Not worth hiring.

b) Some candidates attempt but complete it poorly. Not worth hiring.

c) Some candidates cheat, at which point it is worth asking follow up questions about their assignment. We did this with a recent candidate who then did not bother to respond to our email about the assignment. Not worth hiring.

d) Some candidates upon hearing there is a technical assignment suddenly withdraw their application, where previously they were showing A LOT of interest. Probably dodged a bullet.

e) Some candidates do extremely well, comment their code and in one or two occasions provide documentation. Worth hiring.

Negatives of approach

1) Application drop outs from candidates being put off by the take home task makes it longer to find someone suitable. BUT on the flip side positive for the company since it reduces the probability of a bad hire which is dangerous.

2) Can't always tell if someone has cheated, but that's why it is often backed up with a technical phone interview.

Outcome of this approach

One of our employees who I hired using this approach has turned out to be a star employee. He is still working for us after over a year. He is reliable and technically talented.

bobo2000
  • 12,030
  • 15
  • 46
  • 81
  • 1
    So you based your whole hiring strategy on one guy's performance. Wow... – BoboDarph Oct 24 '18 at 12:57
  • 1
    Exactly why should I be motivated to do free work for your company? Hire me and you'll find me working to help the company. Put too many time-consuming things to do in applying and I'll find another job and work to help their company. Realize, employers, that very few people are interested in working for you specifically, and that the sort of developers you want can find jobs that are easier to apply for. – David Thornley Oct 24 '18 at 15:18
  • @DavidThornley this thread has run it's course. It is about managing risk, if people can't be bothered to do a 30 minute coding exercise, then they are clearly not motivated enough to want to role. Also, as previously mentioned coding exercises are more applicable to people with unproven track records; interns, graduates than experienced hires with bag loads of experience. – bobo2000 Oct 25 '18 at 13:54
  • 1
    @bobo2000 You seem to be missing my point. Suppose I'm looking for a job. I want a job. The odds are that I'm not interested in working for you in particular, but if hired I would do a good job for you. The more time I spend on your application process in particular, the less time I have to work on my overall chances of getting a good job. Should I take your test or apply to two more jobs? Now, if this were a central testing location that would be used by many companies, it would be worth my time., – David Thornley Oct 25 '18 at 18:21
  • @DavidThornley a lot of companies have programming tests as part of their application process, also from experience the best candidates tend to be the one's who take the time to do them - and there will be many who will. – bobo2000 Oct 26 '18 at 12:51
  • @DavidThornley to add, you may think this is an unfair recruitment process, but the cost to replace an unproven developer which turned out to be a bad developer blagging it is much more expensive then just not hiring them in the first place. – bobo2000 Oct 27 '18 at 11:50
  • 1
    @bobo2000o Fairness is not the point in recruiting. You want to give good people a good hearing in order to decide, but actual fairness to candidates isn't even tried for generally. I think you could get a better hiring pool without the takehome, and test their basic competence in a half hour or so under supervision. If you can't tell whether someone's basically competent, you're sunk anyway. It doesn't matter to the people I hang around with. We'll get jobs. It matters to you. – David Thornley Oct 29 '18 at 01:22
  • @DavidThornley well every person I have recruited using this approach turned out to be a good employee based on practical experience of doing it. In contrast, if I never had any sort of technical screening process, it would have been costly for the business. – bobo2000 Oct 29 '18 at 13:40
1

I'd send them to an online quiz where you can filter out people who have no clue. At least you'd have a sense whether they know what they're talking about.

Early in my career headhunters told me "you're up against people who read the box and put an application on their resume." I assume this may still happen to the young and naive, but once you get trashed in a few interviews you learn this is bad advice...

Tim
  • 231
  • 1
  • 7
  • "read the box and put an application on their resume", what does that even mean? – hkBst Jul 23 '16 at 08:31
  • Back in the days when you had to go to a store, people would read the summary about what the software did (which was on the back of the box) and put it on their resumes. These days it'd be comparable to visiting the web site or talking to someone about it and putting it on your resume without ever using it. – Tim Jul 24 '16 at 13:24
  • Online quizzes are an offence to me. Your getting paid to interview, I am not getting paid to take free tests. – SuperUberDuper Dec 07 '16 at 08:26
  • 1
    I'm not paid overtime to spend my nights and weekends cleaning up after these people either. – Tim Dec 08 '16 at 14:30
-1

I recently got a take home test. It was full blown application that needed to connect to a socket server that had to simulate a slow feed. Client had up update dynamically, be able to cancel the feed, and write and read XML.

I kind of want to learn sockets anyway as I am thinking about using them for a poker server I am writing.

I wanted to learn XMLreader and XMLwriter.

At first I thought forget it. But then I saw it as a chance to prove what I can do. I don't have a CS degree so I miss some theoretical stuff. They asked what are the 3 pillars of OOP and wanted to say who cares.

My message is people that want the job should complete the test.

paparazzo
  • 36,763
  • 7
  • 70
  • 118
  • What do you mean by "wanted to say who cares"? – Peter Mortensen May 27 '22 at 06:46
  • That is a cliffhanger! What happened next? Did you get the job? Did you learn something that you have later applied? Does "3 pillars of OOP" imply they didn't care about the result of programming task? What did they say about the completed programming task? – Peter Mortensen May 27 '22 at 06:46