The Myth of the Unbiased Review

Recently, I have a seen a lot of chatter on the ‘net about the objectivity of fountain pen reviewers. There was a bit of a kerfuffle on the FPGeeks forum a few weeks ago, several postings on Reddit, emails exchanged between reviewers behind the scenes, and comments (many of which were poorly spelled, missing punctuation, and all in caps) left on YouTube videos. It is clearly an issue that many people are paying attention to right now. And therefore, it is an issue I’d like to address.

The gist of these complaints seems to center around reviewers receiving pens, ink, and paper from manufacturers or retailers, and how those review items impact the objectivity of reviews. While I can understand, and even share, these concerns, they are indicative of an attitude about FP reviewers that isn’t quite accurate.

I was recently tagged in a comment on the video of my reviewer friend, SBREBrown. It said, in essence, “Your job is not to civilize the Internet, it is to review fountain pens.” (This in regard to my recent crusade to muffle the disproportionately loud voices of online trolls.) While I understand the intent behind the comment, it was factually incorrect. My job is not to review fountain pens. My job is to manage the development of a large-scale corporate website and the related mobile apps. My job is narrate audiobooks in my free time. My job is to take care of my house and my dog. And, if I choose to accept it, my job is to find a significant other so I don’t spend all of my remaining years of life alone. (But that’s an entirely different blog post.)

Reviewing fountain pens? It’s something I do for the fun of it, because it gives me a chance to share one of my hobbies, and because I’m an attention whore. It’s not a job. What little I get “paid” to do it doesn’t even come close to covering my costs, let alone compensating me for my time. So, clearly, not a job.

That’s the conflict that many FP reviewers find ourselves in. We’ve landed in this grey area between hobbyists and professional reviewers. There is an expectation for us to put out reviews of a wide variety of pens at regular intervals, yet nobody is paying us a salary or giving us a corporate card to acquire review materials. For the most part, that’s fine. I’m happy reviewing the pens I purchase for my own personal use. But that means the kinds of pens I review are going to, naturally, be skewed toward my own purchasing preferences. I don’t have the money to buy pens I don’t want, and I’m not willing to bankrupt myself in order to get the materials I need to publish 1-2 reviews a week.

So, when we’re approached by a manufacturer or a pen retailer and offered review pens, inks, or even sponsorships, most of us are happy to entertain those offers. Well over 90% of the costs for producing season 1 came from my personal bank account or credit cards. Season two, it’s closer to 75%, thanks to many generous donations from viewers, and a couple of sponsorships from pen retailers. This can be an expensive hobby, and if the choice ever comes down to me having to jeopardize my ability to retire someday simply so I can record another pen video? Well, I hate to break it to you, but The Pen Habit will be SOL.

The concept of a reviewer getting free access to the items they review is not new. Movie reviewers get free access to the movies they review. Book reviewers get free copies of the books they review. Game reviewers get free access codes to games before they are released. Tech reviewers are sent demo units to review. The internet is littered with “beauty” bloggers who get skads of free cosmetics. Does this impact objectivity? I think that it certainly can, but probably not as much as you may think. You know why?

BY DEFINITION, A REVIEW IS SUBJECTIVE. IT IS BASED ON THE TASTES AND PREFERENCES OF THE REVIEWER. THE IDEA OF A TRULY OBJECTIVE REVIEW IS FALLACIOUS.  An overview goes over the statistics and properties of a pen, but doesn’t include any opinion. This is not what I do, or want to do. I review pens. A review often includes an overview, but also includes the reviewer’s opinions about the pen. Yes, receiving a free pen can have some impact on the reviewer’s opinion, since that free pen may change the “value proposition.” We may subconsciously be more forgiving to a mediocre free pen than to a mediocre expensive pen, but I would argue that the bias is slight. And it certainly doesn’t introduce subjectivity into a previously objective review. The best any reviewer can do is be upfront about it and let the reader determine for herself if the bias is high enough to invalidate the review.

At, I do, and will continue to, accept pens for review as well as user donations and  site/video sponsorships. I want to keep doing reviews, and this is how I am getting closer to affording it. I have always had rules about what I will and will not do as a reviewer, but I’ve never fully codified or communicated them. It is my hope that, with this new Reviewer Code of Ethics, you will know exactly where The Pen Habit stands. That way, you can decide if my opinions are sufficiently objective enough for you to put any stock in them or if you need to find another pen reviewer whose standards more closely align with your own desires for reviews. Code of Ethics

The Pen Habit has always included, and will continue to include, disclaimers in all written reviews and videos that indicate when review materials are provided for review—whether from a manufacturer, retailer, or fellow pen-friends. In all cases, The Pen Habit will make every possible effort to not allow the receipt of an item to color my opinion of that item. However, it is my hope that full transparency will help to offset any unintentional bias that may arise.

Review Items from Manufacturers
The Pen Habit will accept items from manufacturers for review purposes under the mutual understanding that the resulting review will be an honest indication of my feelings toward that product. At no point will The Pen Habit accept items explicitly or implicitly in exchange for a falsely positive review.

  • Pens – The Pen Habit will accept pens for review purposes from manufacturers. Following the review, all pens from manufacturers will be returned to the manufacturer, given away to Pen Habit readership, or, in the event I really like the pen and wish to keep it, purchased with my own funds. The Pen Habit does not accept “free” review pens from the companies that make those pens.
  • Ink & Paper – Fountain pen inks or paper products provided by manufacturers, as low-cost consumables, may be kept by the reviewer, free of charge. Review disclaimers will indicate that the item has been provided for review purposes.
  • Other Accessories – Pen cases, pen stands, notebook covers, folios, etc., may be kept by the reviewer if the cost of the item is under $50 USD. Items over $50 USD will be returned to the manufacturer, given away to Pen Habit readership, or purchased directly from the manufacturer.

The Pen Habit offers sponsorship slots for The Pen Habit website and YouTube Channel. The Pen Habit does not accept sponsorships from pen manufacturers or their brands. The Pen Habit will accept sponsorships from pen retailers who sell a variety of brands and manufacturers.

Pen Habit sponsorships may be purchased with cash or with pens and accessories (which will be assigned a dollar value and attributed to the sponsorship account accordingly) at my discretion. Reviews of pens provided by a retailer sponsorship will include a disclaimer indicating such.

The Pen Habit will not review, rank, or explicitly endorse individual retailers. The presence of a retailer’s advertisements or sponsorship on Pen Habit blog posts or videos should not indicate endorsement of that retailer’s products or services.

YouTube Revenue
The Pen Habit currently allows YouTube to insert pre-roll advertisements before videos on the Pen Habit YouTube channel. The Pen Habit does not control which advertisements are shown in these pre-roll ads, nor is their appearance prior to a Pen Habit video an indication of endorsement of that product or service.

The Pen Habit accepts monetary donations from individual viewers via Patreon, PayPal, or Google Wallet. We do not knowingly accept monetary donations from pen, ink, or paper manufacturers, or individuals who work for those manufacturers. Individuals who donate funds do not get to influence the items reviewed or the content of reviews in any way.


If you have any questions about The Pen Habit Review Code of Ethics, please feel reach out to me using the contact form on this site. The “official” version of this code can be found here. (I only want to maintain it in one place.)

  • JD

    Hi Matt,
    As I said in the comment in my blog, I think it’s great that you’re thinking about these issues and have decided to create a framework for yourself and your readers. I do want to push back against some of what you’ve said, and say that I don’t think anyone is genuinely calling for reviews to be unbiased. You’re quite right to say an objective review would be one where we just share the facts and no personal feelings, and such a review would not be particularly interesting.
    Personally, I just hope to see a community where reviewers who receive products (free, discounted, or loaned from companies) are upfront about that and that the companies have a clear expectation that this doesn’t guarantee them a positive review. Six months ago, I’m not sure that anyone was seriously thinking about this, and I’m glad to see that now it’s a topic of discussion.
    On another note, the comment about your job – you may have taken that too literally. I personally find the ‘civilise the internet’ idea a bit quixotic, and can empathise if the commenter’s intended meaning was more about saying that pen reviews are your strength and, recognising the limited time you have, would prefer you to focus on that strength rather than a vague and perhaps impossible task.
    JD (Pen Economics)

    • Hi JD,

      I am getting quite a few calls (on YouTube, mostly) for unbiased and “free” reviews. Most of them are the rantings of a few malcontents, but it takes up enough of time time cleaning up the name calling and vitrol as to be a nuisance. I certainly don’t think that is the overall tenor of the conversation, but there are a few hardliner folks out there making noise that there should be absolutely no relationship between reviewers and manufacturers/retailers.

      As for “It’s not your job to…,” I agree I responded somewhat literally, and perhaps unfairly. A big part of that comes from a bit of a hyper-sensitivity on my part of an expectation I feel (real or imagined) that I owe the world my time and energies to continue this at all cost. There is a certain aspect of losing “ownership” of creations once they are released to the world, and I tend to get a bit prickly when others treat me as a resource to be exploited rather than a person.

      As for the Quixotic nature of “Civilize the Internet,” I don’t disagree. 🙂 Then again, I’ve always loved windmills.

      Thanks for your comments and your well-considered words. I, too, am grateful there is a conversation taking place, and I hope that we can improve transparency between the manufacturers, the retailers, and the reviewers. There is a whole world of pens and inks still to explore, and I want to be part of it. At a certain point, however, the lost opportunity costs of maintaining the rate of spend I’ve established over the last two years will become too high, and I won’t be able to continue.

      On an only tangentially related note, I hope you are enjoying your new Divina! (It was you that just got one, was it not?)

  • Elizabeth Hall

    Hi Matt.
    This was very well said and I think also that sometimes that if you have to choose between the Life of the pen habit. Or having a healthy retirement, it’s an obvious one 😉

    Thanks for writing to me the other day!

  • lessherger

    I’ve been thinking about these things for quite some time. And it’s good to see others thinking of the same topic. I think that the biggest issue in the review world is what I term the “beauty review.” A review scant on substantive information and based on little to no use of the product being reviewed. I see this with pencils being reviewed on pen blogs. The reviewer will barely use a pencil and more than review the pencil itself, review how it looks. While I think the looks of a pencil are important (hence the popularity of the new Blackwings) the core is more important. So when I see a blog where the reviewer has used the pencil in question (or pen) to write a few lines of text, and then deems it a “winner.” I’m mystified as to how the reviewer can actually call that a review.

    In the end the blogger can pump out review after review and still score scads of free product but the blogger undermines her/his credibility by producing reviews of sub par product that are claimed to be “just great.”

    I was recently scammed into buying a 4 color pen (I use them with my pocket notebook and for studying on the train) that all the reviews I read said was “just great” only to get it and it was horrible. It blobbed everywhere and made a mess of my page with ink webs sticking to the tip of the pen. It was junk, very very pretty junk. The blogs I read, I will never trust again, in fact, I took them out of my feedly so that I’d never be misguided by them again.

  • Indira

    Whenever I watch any reviews I take them as entertainment purposes only. I have never seen them as fact, but only as the opinion of the person who’s reviewing. Their time gives me a chance to see new stuff that I’d probably never find on my own. Depending on the situation it could be tips, tricks, or maybe just a new interesting product in general. Should I then be sufficiently entertained to purchase said item, I blame no one but myself if things go wrong. Same thing goes for reviews people write on product pages. If every review left is positive I can’t simply assume everyone is lying when I end up with a less than happy experience.

    I do find it unfortunate though, that people that just want to post for fun and to share their experience end up getting tangled in disclaimers. These folks use a lot of time and money to try to provide content they hope someone could find useful with little or no compensation for their efforts.

  • Boondoc

    Matt, anytime you take something for free, it is not free. You are going to have some bias, even in a small way, because they gave it to you for free. This is true with doctors getting drug samples, politicians getting free lunches, and every other field. It may be a small effect, but the only way to be completely unbiased is to buy it yourself. Not to say you shouldn’t accept the free pen, but realize this fact: it cannot be controlled.

    • I agree. Which is why the best method for dealing with it is to be completely transparent. I personally don’t believe that the effect is big enough to make a significant difference in the way I do my reviews, but at least if I am upfront about where the pens came from, you as the viewer can take it into account when deciding whether or not you’ll trust me.

  • Paul Burberry

    A very well put piece of reference Matt. I find it amazing that the internet has given everyone with access a voice, but a good chunk of those use that voice to bitch out others in the most scornful ways they can ‘think’ of. I think most people who conduct themselves in this way have a complex of sorts where they feel that because there’s a screen in the way they can talk much bigger. Where I’m from if you spoke that way to a bloke down the pub, chances are you’ll be going home with a shiner, a fat lip and feeling a little more humble than when you uttered the offending remark. Thank you for doing the reviews Matt they are very much appreciated by me. I know these videos come at personal expense to you and I’m sure I speak for many fans of yours when I say that what you do provides us with more information allowing us to make more informed decisions on our own personal writing equipment purchases.

    • Thank you for the kind words, Paul. And a huge thanks to the folks who have donated to help me cover these costs. It’s getting much better in that respect, but it’s one of the reasons why I wanted to codify my own personal rules around money. (People get strange when it comes to stuff like that.)

  • Nisse

    I am quite new to the pen world and stumbled upon the comments on fpgeeks before reading this blogpost. Your words kind of expressed almost exactly what I thought earlier. Thanks for the effort, time and money you spend to give an insight in this wonderful hobby. Don’t ever let those comments or posts on any platform ruin something you like or love!

  • Waski_the_Squirrel

    I appreciate your quest to “civilize the internet”. By yourself, you won’t succeed. But, I think you can inspire others to join you and actually have an impact. I’ve been incredibly lucky so far in my own reviewing and have not had any negative comments. However, in some of my other videos, which were little more than, “Check out the video of this cool place I visited,” I got a lot of negative comments. I ended up taking down a few of those videos which, in retrospect, was just surrendering to the bad part of the internet.

    I think it’s important to know when a pen is provided to the reviewer, either by a company or by an individual. However, I agree with you that an unbiased review is impossible. It’s just like unbiased news reporting. Bias creeps in, no matter what. Better to announce those biases. My reviews always involved a Noodler’s Konrad because I’m really in love with that pen. For better or for worse, that gives people a perspective on my taste.

    I appreciate you codifying your standards on reviews. You’re a big enough name that it matters. I think your standards are fair.

    As for bias: all too often I think that is thrown around as a way of saying, “You disagree with me, therefore you must have been influenced.” It is also an easy cheap shot for those who haven’t bothered to do reviews themselves.

    Please keep doing what you’re doing. And, I believe that your quest to civilize the internet is important. The more of us who join in, the more influence you will have.

  • Pingback: Link Love: Atlanta Snafu Edition | The Well-Appointed Desk()

  • Pingback: My Views On Reviews | Write Analog()

  • Clifford Hughes

    You obviously put a lot of thought into this, Matt. But you don’t have to try any harder, We trust you, ok

  • mikey

    Well said Matt. No one should doubt your integrity!