When conducting a PickFu poll, one of the biggest benefits is accessing an audience of people who have no familiarity with your product, logo, book, or whatever it is you’re testing. They approach the question without bias… but as the poll creator, do you?

Avoid these common mistakes and poll respondents will answer openly and honestly.

Mistake 1. Leading words

Your question may include a positive or negative bias — words that consciously or unconsciously lead the respondents toward a certain kind of answer.

Examples:

• How much did you enjoy this YouTube video? (positive bias – implies that the viewer enjoyed it, and leads respondents to answer more favorably)
• Should responsible parents vaccinate their children? (puts respondents on the defensive by insinuating that parents who do not vaccinate are irresponsible)
• A recent poll found 80% of Americans disagreed with this government policy. How unhappy are you about this policy? (negative bias – not only is the question phrased negatively, it also includes a statistic that shows many unfavorable views, leading the respondent to feel as though he or she should feel that way, too.)

How to fix it

Remove leading words and phrases and structure the questions as objectively as you can:

On a scale of 1-5, with 1 being the worst and 5 being the best, please rate this YouTube video.
• Do you think children’s vaccinations should be required?
• Please rate your level of agreement with this government policy.

Remember, biased questions lead to biased results. And if you’re using polling to make important business decisions, you want your feedback to be as objective as it can be.

Mistake 2. Making assumptions

Sometimes questions include an opinion. For example, “Do you agree that dog owners should be able to walk their dogs off-leash in the park?” This assumes that dog owners want to walk their dogs without leashes, which may or may not be true. But by including this assumption, you are leading more people to respond in agreement.

Further examples:

• Is your favorite color blue? (assumes the respondent’s favorite color is blue)
• As long as nobody minds, is it okay to smoke indoors? (assumes that nobody minds)
Where do you like to party? (assumes respondents like to party)

How to fix it

Structure your questions around facts, not opinions. Make sure you’re not painting respondents into a corner so that they can only answer one way. In the first example, it would be better to ask, “Do you think dog owners should be allowed to walk their dogs off-leash in the park?” Similarly, these questions eliminate assumptions:

• What color do you like the best?
• How do you feel about smoking indoors?
• What do you like to do in the evenings?

Mistake 3. The double-barreled question

Sometimes questions ask about two disparate things. These double-barreled questions do not make good survey questions because respondents will often only concentrate on the one topic that means the most to them.

Examples:

• How do you feel about our two new flavors, French Vanilla and Hazelnut Cream?
• How have teachers and students responded to the new dress code?
• Which title and subtitle do you prefer?

How to fix it

Double-barreled questions should be separated into two distinct questions and polled separately from one another. When questions are worded similarly in a multi-question poll, it’s also good to distinguish them with underlined words or italics, as below:

• How do you feel about our new flavor French Vanilla?
• How do you feel about our new flavor Hazelnut Cream?
• How have teachers responded to the new dress code?
• How have students responded to the new dress code?
• Which title do you prefer?
• Which subtitle do you prefer?

Creating better PickFu surveys

PickFu focuses on preference testing. This means that in general, you’re testing options against one another, not necessarily open-ended or yes/no questions. Therefore, a question like “how do you feel about smoking indoors?” or “How have teachers responded to the new dress code?” likely wouldn’t get asked through a PickFu poll.

That being said, the same rules apply to preference testing as in general surveying: avoid leading words, don’t make assumptions, and only ask about one thing at a time.

For example, if you’re testing mobile app icons, the simplest unbiased question would be, “Which app icon do you prefer?” To embellish the question with anything else may introduce bias. Sometimes pollsters want to know whether their icons read a certain way, so they’ll ask something like, “Which icon seems more kid-friendly?” While not biased, per se, this question asks respondents to judge your icons differently than they might otherwise.

Because PickFu respondents include comments, one approach to avoid any hint of bias is to ask the simple question, “Which app icon do you prefer?” and then read the comments to see if kid-friendliness is something the respondents took away by themselves, rather than being led to see kid-friendliness or not.

Another approach is to include factual statements (unbiased, of course!) to explain what you’re testing, and then ask the respondent’s preference. For example, “Our app is a game aimed at children under 10. Which icon do you prefer?” These approaches may all yield various results, so some experimentation might be necessary to see what’s most applicable in your case.

Keep it consistent

Consistency is key in testing creative options. If you’re testing book covers, make sure that each design option includes the same information. If you tested two cover designs and each cover had a different title, for example, you wouldn’t know whether respondents preferred the design layout or the title on that design layout. Remember, only test one thing at a time.

Below is an example of an author who tested two covers. However, the two titles are different, only one option included a subtitle, and only one option included the superfitdads logo. Any of these variances may have skewed the results.

This poll would be less biased if each cover included the same title, subtitle, and author attribution. Then, the only test parameter would have been the cover design.

Here’s a better example. This author is testing two cover designs. The title, subtitle, author attribution, and even graphic layout is the same. The only difference is the “Action Plan” stamp. By only testing one thing at one time, this author knows the stamp helps his cover.

Summary

In both general polling and preference testing, it is important to keep these main ideas in mind as you design your survey:

1. Avoid leading words that may sway the responses positively or negatively. Phrase your question objectively.

2. Ensure your questions are fact-based, not opinion-based. Do not make assumptions about your audience.

3. Test only one thing at a time. Do not use double-barreled questions. When creating comparisons between two creative options, include only one test parameter per question, such as layout, title, or color palette.

4. You may need to experiment with the level of specificity in your question. Generally, the simplest form of the question will be the least biased. However, there are instances where you will need to direct the question a certain way rather than leave it open-ended. Just be sure to do it according to the three tips above.

PickFu relies on a number of quality controls to ensure that the people who participate in your polls answer them honestly and seriously. And guess what? YOU are one of those quality controls.

When your poll is complete, you should take the time to read through each comment. If an answer is particularly insightful, use the up arrow to the right of that response to “upvote” it. You can tell us what you liked about the answer, and your comment will be shared with the PickFu team (not with the person who wrote it).

upvotes

Similarly, if there is any response that was inappropriate or displeasing in some way, you can vote it down. Once again, you can share a comment with the team about why you downvoted it, and we won’t invite that respondent back for future polls.

You can submit up to five upvotes and five downvotes on each poll.

downvote

Even more, the conversation doesn’t have to end when your PickFu poll is complete.

Let’s say a respondent wrote something that you need to be elaborated. Maybe you aren’t sure what she meant but would love to know.

PickFu offers the ability to ask a follow-up question. Reach out to a respondent directly and get back in touch regarding her comments. Simply click on the speech bubble to the right of the response.

follow-up-question

If you don’t see the upvote and downvote arrows or speech bubbles, make sure you are logged in with your PickFu account. Only then can this power be unlocked!

Has this feature been helpful to you in your polls? Let us know in the comments!

Exactly who likes your product or design should never be a mystery to you. Knowing your audience means understanding their needs and desires, and knowing how best to address them.

On PickFu, you always know who answers your polls – each result includes a demographic breakdown of gender, age, income, ethnicity, and education level. But you can also target certain demographic groups so that only certain subsets of the population respond to your poll.

“Your audience gives you everything you need. They tell you. There is no director who can direct you like an audience.” – Fanny Brice

Mobile Device Platform

One of the most practical demographic segments PickFu offers is targeting iOS or Android users. If your app is only available on one platform, why poll those who couldn’t download it if they wanted to? This Android game wanted to see what users thought of a new mobile app icon. The poll encompassed Android users between the ages of 18 and 34:

app-icon-testing

PickFu also offers the ability to target Mobile Gamers.

Reading Preferences

Authors use PickFu extensively to test book titles, subtitles, cover designs, and blurbs. Knowing this, we created several categories on PickFu so that authors could better target readers. You can poll based on the type of reading (fiction or non-fiction) readers prefer, and the number of books they tend to read per month.

When writing effective book descriptions, a top tip is to use the first sentences of your blurb to hook a reader in. One author used PickFu to poll females who preferred fiction to gauge which opening sentences were more intriguing.

reader-poll

Vegetarianism

If your book is aimed at a niche market, why poll those outside that niche? One author whose cookbook included kosher vegetarian recipes polled 50 vegans and vegetarians. They overwhelmingly preferred the colorful, more poetic option over the straightforward title.

demographic-targeting-vegetarians

It would have even been possible to target only vegetarians of the Jewish faith on PickFu, though it would have taken longer to complete the poll. This author decided not to keep the focus that narrow. After all, the kosher recipes could still be enjoyed regardless of religious beliefs.

Income

When you’re launching a boutique product, it makes sense to ask opinions of those in your target market. This line of organic skincare products polled women who made over $60,000 to see which logo they preferred:

logo-poll

Age

Perhaps your product is aimed at the youth market. This upcoming line of streetwear wanted to see what males under the age of 50 thought of its logo design:

poll-young-males

These are just some of the ways the ability to laser-focus polls has helped our customers. Who is your main audience? Do they share certain traits? Are there even more segments that would be useful to you? Send your thoughts to us @PickFu!

As Americans, we often scratch our heads at the legions of international soccer fans who can live with a tie. A score like 2-2 (or even worse, a 0-0 draw) is so inconclusive – we like the clarity of winners and losers. When a game ends tied, how are we supposed to feel?!

But a tied game is still hard-fought. The players still put it all out on the field, exert huge amounts of effort, and in the end, are shown to be evenly matched. Alan Jacobs wrote an impassioned defense of ties a few years ago in the Wall Street Journal. In it, he writes,

Since scoring is so rare, many matches end 0-0 or 1-1. And this is something that we soccer fans don’t just accept about the game: we love it. We love that scoring is so darn hard, that, most of the time, many interlocking pieces of game action have to fall into place just so in order for the ball to make its way into the back of the net. We want it that way.

This is a good lesson when your PickFu poll ends in a tie. It’s difficult to score big with an audience. You probably know (or are) someone who will only drink Coke or only drink Pepsi. These are passionate, insistent customers. But you also probably know someone who doesn’t have a cola preference, or even can’t tell the difference. (These people are crazy, by the way, because obviously Coke is superior).

When your PickFu poll ends in a tie, it could mean that the audience saw little discernible difference between the options presented. Or it could mean that equal-sized segments of the audience showed a strong preference for their choice. In either case, a tie is information that in itself can be valuable.

What to Take Away from a Tie

First of all, a tie could mean your options are equally preferable. Hooray for you! You offered good options. You could interpret this to mean that whatever avenue you go down, a segment of your audience will be amenable to it.

One PickFu pollster wanted to know which pricing model customers would prefer for a travel adventure package. The first choice was to pay between $25-35 for the itinerary and pay separately for the activities, and the second choice was an all-inclusive model. In a poll of 50 respondents, 26 preferred the first, and 24 preferred the second. While not a numerical tie, it’s a statistical tie without a clear winner.

tied-poll

A reasonable conclusion might be to offer both options to customers. That way each client can pay according to his or her preferred pricing model. This is similar to what we do here at PickFu – offering both à la carte and subscription options because everyone’s needs are a little different.

Dig deeper

Another way to get insights is to dig deeper into the comments. Are respondents confused about something? Is there an element that consistently causes them to react negatively? See what you can glean from what people said, and use this knowledge to choose the winner or to iterate on a new version — a version that, perhaps, might become a clearer winner.

When Michael Cowden, another PickFu pollster, wanted to name his mobile game, he tested two potential names: Outrun the 80s and Super 80s World. Super 80s World won, but only narrowly – by a margin of six votes. Should those six votes be decisive?, he asked himself. In the end, the decision was yes.

Based on the comments, those who voted for Super 80s World were closer to his game’s intended target. As he wrote on his blog, “I could immediately tell that Super 80s World communicated the game concept much better. The folks that liked that name got what the game was about and, more importantly, wanted to play it!”

tied-poll-2

When should you retest?

While Super 80s World ended in a close result, another game with two potential names ended in a perfect tie. A poll of 50 mobile gamers split right down the middle when choosing between Squid Attack! and Squid Squad.

tied-poll-3

Many respondents liked the alliteration of “Squid Squad” and thought it was fun to say. But respondents repeatedly commented that a “squad” suggests cooperative play (which the game is not), while “attack” sounds more like a player vs. player game (which it is).

So what should this pollster do?

Personally, I would choose the option that reflected the player vs. player game format. But if the decision-maker was still unsure, another idea is to add more respondents. With a larger pool, you’ll receive more comments, and these additional voices may help clarify the road ahead. If a poll is not statistically significant, adding more respondents may also help achieve that significance – which means that the results are likely to be replicated were the poll to be run again. More respondents can also reinforce what the first pool of respondents said, and affirm that many people agree with them.

When to re-examine your options

A tie may also mean that the options you tested were not distinct enough. For instance, one PickFu pollster tested two logos that were identical except for the color. With 100 respondents, 50 chose orange and 50 chose blue. Many comments gave reasons such as “Prefer orange to blue,” or “Blue just looks better in general.”

Back to the Coke/Pepsi analogy, some people just like Pepsi. Others are right and love Coke (just kidding – sorta). However, most people simply don’t feel strongly one way or the other, such as these comments in the orange vs. blue debate:

• “Not very interesting a question, given that they’re the same logo just palette-swapped, but I like the blue more. But it really doesn’t matter. They’re pretty similar.”

• “Not really that different, I slightly prefer B because it’s blue and I like that color better. Everything else looks the same though.”

• “I prefer the color scheme of choice A, however it is not overwhelmingly better”

It may sound harsh to say it’s just a toss-up. Should you really just flip a coin with an important decision like this?

Not necessarily. Respondents in this poll also gave comments about what they read into each color, and you might take some of their ideas into consideration.

“The orange is more aggressive,” wrote one male. A female respondent agreed, writing “I find the orange logo (option A) much more preferable because it feels more action-oriented and looks cooler than a blue logo, which makes me feel calm.”

Another person surveyed wrote, “The blue seems cooler, more masculine,” while yet another wrote, “The orange makes it stand out a little more. It also seems like it would be a little more gender neutral.” “Blue is a cooling, calming color,” one male said. “It hints at ‘newer tech’, and makes the user feel like theyve made a sound decision.”

The feedback may be contradictory, but taken together, it may also add confidence to the direction you were already heading, or open up new questions you hadn’t considered. The ultimate tie-breaker in a decision like this is you. You can use your own discretion about which points reinforce your position, and those that make you think a bit differently.

How do you feel about ties? Have you been torn between two options before?
Let us know in the comments!

An attractive photo, a great layout, a memorable logo or app icon – these elements are often touted as the keys to optimization. Indeed, a well-converting site, ebook, or app will need them all. But sometimes we tend to underplay or altogether overlook the importance of copy testing – and to our peril. Whether it’s description copy of an e-commerce product, the subhead of a new book, or the elevator pitch of a growing startup, words matter.

Recently, PickFu increased the number of characters you can include in your test text block to 1,000. That’s the equivalent of roughly 200-250 words. Here are some great ways to use this feature for copy testing to improve your business: Continue reading

99designs is a great site for startups and small businesses – for just a few hundred dollars, you can launch a design contest for a logo, WordPress template, PowerPoint deck, signage, and more. Graphic designers around the world compete to win, you provide feedback, and after seven days, you pick a winner.

Here at PickFu, we crowdsourced our own logo using 99designs. Once the contest began, however, something became clear: even though receiving over 350 designs was valuable from a cost perspective, choosing a winner among them all was beyond overwhelming.

“We’re programmers, not designers,” said Justin Chen, PickFu co-founder. “Other than my own visceral reaction, it was hard to judge the value of all the colors, typefaces, and icons.” Continue reading

A short while ago, I wrote about Twitter’s new polling feature. I was excited when the feature was rolled out to me so I could try it myself.

Now, I’m not the world’s biggest Twitter user. I’m certainly no celebrity. But I’ve been active on Twitter for over five years and at the time of this writing, I have 423 followers. (Want to boost my ego and add to the total? Follow me @kimkohatsu!) My 400+ users are comprised of professional contacts, brands I’ve mentioned or contacted via Twitter, and some friends and family. According to a 2012 study (the latest I could find), the average Twitter user has 208 followers. I’ve got over twice as many — not bad, right?Continue reading