I love TechMeme. Before TechMeme, I was a big fan of Digg. I still like Digg, but TechMeme is now my number one news source. It’s incredible. Before you read this post, please go check it out.
Did you check it out yet? If not…
Really, it’s important that you check out TechMeme before reading this post…
So now that you’ve given TechMeme a spin, you have probably noticed that it ranks the most interesting content in technology happening on the web RIGHT NOW, organizes clusters of RELATED posts, and presents the clusters in order of INTERESTINGNESS. There are numerous theories on how TechMeme actually works and a number of people have tried to reverse engineer the algorithm. TechMeme founder and sole employee, Gabe Rivera, gives us a peak into how the system works, here.
My extremely non-scientific, back-of-the-envelope summary of how TechMeme works:
1. Seed the system with “white list” sources. Looks like Gabe has picked the seed sources.
2. From that seed group, explore the internet for cross-linking behavior.
3. Develop an algorithm that ranks URLs (stories) based relatedness (to form clusters) and interestingness (by cluster and by story within a cluster).
4. Display results. Treat your algorithm like a hypothesis and continuously use A/B testing to empirically test your hypothesis. I haven’t seen anyone talk about this last point, but it seems like a really good idea.
New Idea: The Magic Meme Machine (MMM)
The idea is to create a new type of feed reader based on TechMeme’s approach. Let’s do it.
1. Seed the system. Consumers load their favorite RSS feeds into The Magic Meme Machine. We are going to create three views: the view of your PERSONAL feeds, a SOCIAL view of your friend’s feeds, and a COMMUNITY view of everyone’s feeds.
2. Develop an asymmetrical social graph. I love the way FriendFeed leveraged my Facebook friends to turbo-charge my FriendFeed subscriptions. They then allowed for asymmetrical subscriptions. For the MMM use-case, asymmetrical connections is the right approach.
3. From the seed group (PERSONAL, SOCIAL, and COMMUNITY), explore the Internet for cross-linking behavior.
18 Comments
May 28, 2008 at 9:19 pm
Hey Mike! Bigger font and/or fewer words, OK?
Only VCs genuinely think that “finding enough new crap to read on the Internet every day” is a fundable problem. This is because you spend way too much time with other VCs and rich geeks — two classes of people who see filling their hours with endless surfing as a lifestyle choice. Unfortunately, there are only a couple million of those people… and they’re all already hooked up with a news aggregator for their Mac gossip. Kthxbye!
May 28, 2008 at 9:23 pm
Oh, and Mike? Learn about “overtraining” before you go on about any sort of empirical testing regime.
May 28, 2008 at 9:23 pm
I guess for me the problem is at step #1. “Consumers load their favorite RSS feeds…”.
I think the power of digg, techmeme, reddit, slashdot etc is that there is a default view that I can pretty much trust reflects what is happening now and is interesting. I don’t have to do anything beyond load the page. I suppose there is personalization if I want to go further, but I don’t.
So my first comment would be: Make it work great for the casual visitor with near-zero-effort. All personalization should be optional, including social networking.
Something else I’ve been thinking about is providing more granularity to user ratings. So instead of simple thumbs up/down, providing ratings like: quality, agree/disagree, maturity-level, spaminess, etc. In fact, a developer at my company did a prototype of a reddit-like site that incorporated such ratings, but it was never fully completed. See http://newsgo.osc.co.cr/
May 28, 2008 at 9:30 pm
Troutgirl, great to have you here
So now that you have trashed this idea, can you go out and build this for the VC community? We are time-constrained and helping us make better use of our time will make the world a better place. Really.
Mr. Libby,
Excellent feedback. On user ratings, don’t you think that the best systems force users to rank order their favorites from 1 to N rather than any number / thumbs-up/down system?
You are another guy who could build this thing in a night. And since you and Troutgirl are buddies, I challenge the two of you to go out and build competing products.
Who’s great and who’s just good
Love the feedback from both of you. Thanks.
May 28, 2008 at 11:15 pm
Mike, my new #1 read: http://news.ycombinator.com/
May 28, 2008 at 11:28 pm
I had a similar idea (that for now I call “Personal Content Matcher” or PCM for you ;-)) in which users define different tastes in content by providing a list of sites that reflect that taste. Sites are profiled on multiple metrics (such as content type, writing style, target audience, niches and even site layout), and matches are performed against the list of sites provided by the user. The idea is not to match other users’ tastes but site characterics as a more relevant matching criteria. The profiling of sites is performed by the community in a gamely like interaction (not too dissimilar to what google is trying to do with the google images labeler).
The concept is similar to the music matching service, Pandora - with digital information instead of music and content sites / creators instead of albums / musicians.
May 29, 2008 at 2:14 am
Thanks for the heads-up Brian. Looks very cool. Impressive what Y Combinator folks are doing. Just awesome.
Eran, love PCM. Are you building it? I would love to be an early alpha tester
May 29, 2008 at 5:32 am
Mike — what you’re thinking will be possible in a few months. I’m a student working for Google Summer of Code building an open-source memetracker. There’s no working code yet but it’s coming. Read my proposal on my blog for all the gory details:
http://kyle.mathews2000.com/blog/2008/04/04/drupal-memetracker-module-my-google-summer-of-code-application
Using Drupal and the memetracker module I’m building, you’ll certainly be able to do what you describe.
There isn’t code but check http://drupal.org/project/memetracker over the next 3-4 weeks for the first alpha code that’ll be uploaded.
I’d love to hear from anyone interested in collaborating on building an open-source memetracker. Use the contact form on my blog to get hold of me. Also, for any developers out there — I posted my plans earlier for the architecture of the memetracker library. Offer any feedback you might have here:
http://groups.drupal.org/node/11820
May 29, 2008 at 6:05 am
Joyce: um, perhaps a bit harsh on mike?
he’s certainly not your avg VC to be bucketed in
with other db sheep
(or is there some history here I’m unaware of?)
regardless what happened to ‘light a Startup candle’ vs ‘rail against the VC darkness’ eh?
you can rip him a new one, just give us some alternative solutions / ideas…
(anyway I’d rather read about people trying on new ideas with 1 out of 5 making sense, than evertyhing being safe & rational … who gives an eff about that?!)
May 29, 2008 at 11:17 am
TechMeme and ‘bike-shedding’…
The problem with TechMeme is that even though the name suggests it is about Tech - it is really about politics (in Tech). Politics is important, even in Tech, sure - but this only amplifies the importance - while the ideal TechMeme would rely more on…..
May 29, 2008 at 4:10 pm
Thanks Dave.
Joyce (Troutgirl) and I worked together at Epinions, so we’re old friends. I had an IM chat with her the other day and I asked her to give me her best shot in the comments section. And, of course, she happily took me up on that challenge
I can’t wait to see what Joyce comes up with next… Would love to hear her ideas, because she has some pretty great ones.
May 29, 2008 at 4:15 pm
PS. I agree wholeheartedly with Dave’s comment that it’s better to put out crazy ideas, many of which are terrible, rather than play it safe. And trust me, I have plenty of crap to share with the world. Hopefully I will come up with something good, even if by random chance since the volume will be so high
May 29, 2008 at 10:22 pm
@Mike
Yep, I will be building it in a month or two (a little short on time right now since we are close to release at my start-up). I’ll be sure to send you an invite
May 30, 2008 at 5:25 am
I want someone to send me just one email (email is the new RSS reader, don’t you know) of the top 10 news I need to know that day so I can feel cool & informed. I have no personal preferences - I want whatever the cool kids are drinking these days. I am sick of subscribing to newsletters at each & every site, because I don’t know if these are the right sites to track. I would gladly pay $9.99 per month for such a service, and so would a lot of other lazy people (I just paid $350 for iRobot’s Roomba to do my carpet cleaning - whats another $10?)
Y
May 30, 2008 at 3:47 pm
aha, that would explain the “history”… and the ‘kthxbye’ comment
(figured she was being her snarky self, but just wondered if i was missing something)
joyce is smart, funny, and certainly not short on biting criticism (occasionally tho, it has bitten back
plus i love:
“Only VCs genuinely think that finding enough new crap to read on the Internet every day is a fundable problem”.
ROTFL.
May 30, 2008 at 3:49 pm
one note: i do agree with her… hire a designer for this blog dude.
try http://99designs.com/ (formerly Sitepoint Contests)
we got our Graphing Social Patterns logos & design ideas there, found a designer in Serbia who does great work.
world is flat, indeed.
May 30, 2008 at 9:33 pm
Awwww, Dave :-(. Here I was feeling all frisky and sportive, unfettered by my normal heavy burden of having to be sweet and well-behaved in public… and not only are you applying the smackdown, but you’re accusing me of “biting criticism”! Way to encourage free and frank exchange of views, dude. Let me go back to my cave now, and watch a few Zuckerberg interviews to hone my “boring and lame but not critical” skills.
Yah, Speiser and I go back to literally my first day in Silicon Valley. And he’s always appreciated brutal honesty… no one keeps it real and rapid more than Mike.
May 31, 2008 at 9:07 pm
Dave and Joyce — it’s good to see that the two of you have joined forces to give me consistent critical feedback on my site usability
Leave a Reply