Well if you ask me...

Twitter Search Ranked By ReTweets

The existence of any ranking system based on retweeting, officially sponsored by Twitter or not, will have a negative impact on the Twitter experience. It will change the way people tweet. IMO, that is a bad thing.

Yesterday [I posted][1] about filtering Twitter search results and I left off with a quote about Google’s PageRank which I think is apropos to the discussion of ranking tweets based on the number of retweets (or twitterers based on the number of tweets being retweeted – or sheer number of retweets from a twtterer)

PageRank stopped working really well when people began to understand how PageRank worked. The act of Google trying to “understand” the web caused the web itself to change. -Jeremy Zawodny

The importance of this cannot be understated. Google, once a company which wanted to provide relevant search results by understanding the relationships between webpages, is now a company that dictates the decisions of millions of webmasters. They don’t do this by decree, it is simply a function of people’s desires to be found on the web.

Google’s idea was a good one – use the structure and nature of html pages to rank them in terms of importance (“authority”) and then use that as one criterion for determining the relevancy of a page. This good idea led to at least two inherent problems.

First, it lent authority to webmasters who deserved no such thing. Years ago, I had several web pages with decent pagerank. As a result, I could post a page about almost any subject and my page would rank higher (perhaps) than a real expert on the subject. As I noted yesterday, others have pointed out this problem. Of course, the web being what it is, my page could be usurped by the proper authority on the subject – IF the web behaved as it used to. This brings us to the next problem.

The web now behaves differently because of Google. I am not complaining about this. I am not unhappy about Google’s existence – quite the opposite. I am simply making an observation that many others have made before me. Webmasters now have some understanding of PageRank and now decide how to construct websites to optimize it.

A retweet ranking system will cause the same thing to happen with Twitter. It will undoubtedly cause people to change the way they tweet. Many users will:

1. think twice before tweeting something that is not “retweetable”
   There go all the fun updates about feeding the kids, eating breakfast, etc. I know many of you don’t like those, but I’d argue that most users enjoy this Twitter “nonsense.”

2. compose more retweetable tweets

Now more tweets will be like those annoyingly profound “updates” on Facebook – yuk!

1. ask for retweets
   I propose that any ranking system automatically take this into account and not credit for any tweet which contains the phrase “please retweet” or similar.

2. retweet less
   Many will hesitate to give away authority to others, or at least be more careful about who they give it to. This is very similar to PageRank leak.

3. seek more followers
   While number of retweets is certainly better than a straight ranking based on number of followers, it is still very much a function of the number of followers. Ideally, those with more followers need to be retweeted less but I assume that doesn’t hold up in practice. More followers will probably lead to more retweets.

Obviously, I am assuming a simple ranking system which rewards users whose tweets have been retweeted. I understand that a system can be put in place that ranks the tweets and not the twitterer. I’d argue that all points above still hold. I also understand that a more complex formula can be devised, but the point is still that people will tweet differently. Imagine a system that rewards one for retweeting other’s tweets. This will lead to a Digg-like echo chamber where perhaps very few tweet original content.

As I said in yesterday’s post, I am in favor of filtering my Twitter search results. I’m just not a big fan of ranking by retweets. More on that later. [1]: /twitter-search-by-authority/

Twitter Search By Authority

Loic Le Meur proposes that we need a “search by authority” feature for Twitter and then recommends that number of followers be used as an indication of that authority. Michael Arrington agrees.

Both took some heat for this proposal. Most of it seemed to revolve around the idea that authority equals number of followers, but I think the discussion is more interesting than that one issue.

I can think of at least four questions this raises.

Do we need to filter search results at all?

I think the answer is yes – sometimes.

Gary Arndt comments

Microblogging doesn’t lend itself to search. Period.

I agree, but I think he is overstating a bit. A Twitter search is not the place to do in depth research on a topic, but the fact that the results are “all over the map” doesn’t mean that one can’t find useful results. The timeliness of the results alone can be very useful. There is simply no doubt about this.

Several commenters pointed out that we should all just enjoy Twitter the way it is. I understand the argument here but there is nothing wrong with discussing new ideas and improvements to existing technology. I enjoy Twitter the way it is and plan to use it the way it is, but I don’t mind that others would like to make it more useful.

Most of the time, I want my Twitter search to return more results than I can possibly process. There may be times, however, that I’d rather filter out some results. Having a mechanism to remove those results voluntarily is always a good thing. I use the minus operator all the time to narrow down my searches. If I could filter based on other criteria voluntarily, that would be great too.

Should we filter based on authority?

I hesitate to answer this honestly because I think many of you will stop reading here,
but I think the answer is yes.

When I read Loic’s post the other day I was as irritated with his tone as many of you, but I do think his point is a valid one. What he was recommending, according to his clarifying comments, is an option to filter based on authority. While I might not use this feature as much as some, I think it’s a good idea.

I tend to agree with Steven Walling who comments

for many, many people, the point of Twitter search is to find things said about a topic of interest regardless of how influential the person speaking is.

and Joe Duck, who adds

In fact I think it would be interesting to reverse the algo you suggest – I’d rather hear from some Grandmas in Peoria about their iPhone experiences than from Jason Calacanis about [groan] the wonders of Mahalo.

I could not agree more with these two comments. Our family had so much fun on the night of Hillary Clinton’s concession speech.^* Terri and I both punched slightly different search criteria into Twitter Search and we read some of the more interesting (and family friendly) comments aloud. It was a good time had by all. None of us cared one bit if the people spouting off had any kind of authority at all. That wasn’t the point.

I may not be interested in search by authority, but I understand why some people want “authority” opinions on a topic from time to time. Some of these might be egomaniacal, but that should not keep us from using an “authority search” for legitimate purposes, should it?

This brings us to the most difficult question of all – though perhaps the least interesting to me personally.

How does one measure authority?

I think the verdict is clear from the weekend discussion – the number of followers is not a good measure of authority. Many people weighed in on this one but two comments sum it up for me:

And if authority is what we’re after I doubt if number of followers equates to authority. Too many really smart people have very few followers. -Dave Winer

Popularity does not equate to accuracy. If it did and Twitter existed 2000 years ago; the earth would still be flat. -Josh Jonte

As far as I’m concerned, that settles it. I’ve used Twitter since 2006 and my experience tells me that the number of followers is not a valid measure of authority.

Nitin Borwankar and Taylor Barr also touch on the problems with such an approach and hint at the possible solutions. I will have more to say about those later.

So, is there some other measure one can use to judge the “authority” of a tweet or twitter user? Scoble and others recommend we use some other metric – retweets, followers/following, someone even proposed

~= #followers/(#faved + #RTs)

My interest in finding the perfect formula for this is negligible but I do think that we’ve discovered one other measure that won’t work – retweets.

Once again, I think two comments sum it up best:

A sycophantic echo chamber is as stupid of a metric as # of followers. -T. Luoma

All these popularity contest-like sites and lists full of people echoing each other all the time just seems to take some of the original spirit away from Twitter. imho. -kosso

Michael Arrington points out that “This is exactly what Technorati does with blog search” and Robert Scoble points out that this is just what Google does. This, by itself, is reason to question it in my opinion.

Scoble goes on to point out that according to Google he is the number one result for “recession 2008” “How did that happen?” he asks.

Jeremy Zawodny points out a similar issue in 2003 and correctly (IMO) diagnoses the problem a few months later:

PageRank stopped working really well when people began to understand how PageRank worked. The act of Google trying to “understand” the web caused the web itself to change. -Jeremy Zawodny

I will pick up my thoughts on this tomorrow, but I do want to bring up the last question.

What other filter criteria might be useful?

It seems clear that at least some users would like to filter search results from time to time. What criteria would be useful to you?

^* Terri and my daughter were huge supporters of Hillary Clinton. While my son and I supported Obama from the beginning, we would have wholeheartedly supported Hillary if she’d earned the nomination.

Well, I guess I can talk about it now …

The Mets really let us down this year.

Billy and I went to Shea for five games (and one rainout) this year. All of our games were after the change in management and it seemed as though the Mets were on a roll. We were certain the Mets would not disappoint us as they did last season.

We also took the girls with us for a game against the Phillies. We had timed it just right. The Mets tied with the Phils for the NL East lead on Sunday 7/20. Both teams were off on Monday and our game was for Tuesday. The Mets beat up on the Philles for more than 7 innings and then it all fell apart. It turned from one of the best days of my life to one of the worst – all in the space of an hour or less.

It was a microcosm of the season really.

One of the games we did get to attend turned out to the last game played at Shea. We’d planned for it to be the last **regular season **game. We were perfectly willing to let others enjoy the last games if they were in the playoffs.

I didn’t expect the Mets to do much in the playoffs but I wanted them to prove to me that they would not collapse again.

I was in Philly today. I left by 3 so I didn’t get caught up in the mayhem for the 4:35 game. I have to tell you that I was jealous. All those fans enjoying their team’s success. I was a little sick.

Watching game two now and the Phillies are in the midst of a rally. Ugh.

Go Dodgers!

Interesting Place to Pick Back Up

I decided to start my new blogging routine today and I noted a link in my last post – “How much will the Mets miss Tom Glavine?”

Not much, huh?

Anyway, I plan to start with a few book and DVD reviews since I am changing the way those display on my homepage. After that, maybe I’ll have something to say about the election.

Math and Baseball

Two of my favorite things.  It can’t get much more fun than this!

How Much Will the Mets Miss Tom Glavine? » Mets Geek

I’m Back

After a year and a half of doing other things and ignoring this place I am finally back. I thought I was done with this but I decided I missed it.So, I’m back!

Catalog Choice – Welcome

I do like catalogs but I don’t want to receive them all! Check this place out. Catalog Choice – WelcomeCatalog Choice is the free service that lets you opt-out of unwanted catalogs.

A.C.L.U. May Block Criticism by Its Board – New York Times

The American Civil Liberties Union is weighing new standards that would discourage its board members from publicly criticizing the organization’s policies and internal administration.A.C.L.U. May Block Criticism by Its Board – New York Times

WHAT?!?! I know many people think that the ACLU is “off it’s rocker” on some issues, but I am a big supporter of what they stand for, or what I thought they stood for. I even understand that some corporate entities need to control PR, but it just seem ridiculous that the ACLU would do anything to stifle speech of any kind. I am very surprised to say the least. Thanks to Stop the ACLU

Virtual Talmud: Rick Warren’s Mega-Synagogue

I am a little confused by Rick Warren’s involvement in growing Mega-Synagogues. Not sure what to think actually.

The idea of mega-synagogues with yoga workshops, banjo-playing Jewish rock stars leading mussaf service, and Rick Warren-styled rabbis preaching to 10,000 overworked, overeducated, and over-purpose-driven Jews would sure be a sight to see.Virtual Talmud: Rick Warren’s Mega-Synagogue

Mike Davidson: I Steal Television Shows Because I Have To

As a TiVO lover and a web developer, I had to laugh at this

let me just say that this recorder obeys orders about as reliably as Internet Explorer renders CSS. That is to say, sporadically, sloppily, and at times, without reason.Mike Davidson: I Steal Television Shows Because I Have To</p> Unlike Mike, I stuck with DirecTV so I don’t have this problem.:)

Kobe vs. Wilt

On January 22, Kobe Bryant scored 81 points in a single NBA game. This is an amazing feat which one can’t help compare to Wilt’s 100 point game. A Sports Illustrated article which reminds us of the difference in stature between the two players even prompted some to hail Kobe’s performance as superior given he is six inches shorter than Wilt. While I think this is a ridiculuos idea, I couldn’t help but put together a spreadsheet to determine what it takes to “break the record.” I factored in three point shots while I was at it. Of course, I couldn’t resist an online version as well.:)On a related note, I read another interesting article about a man who gave up tickets to see Wilt’s 100 and Kobe’s 81!

‘Fossil’ hat

Help me out here. Anybody know where this mine in Tasmania is? ‘Fossil’ hat

Bid to give Bush the boot

“Therefore, the voters of the town of Newfane ask that our representative to the U.S. House of Representatives file articles of impeachment to remove him from office. ”All in favour?: 129. All opposed?: 21. Meeting adjourned.Bid to give Bush the boot

Wow! Thanks to Fountainhead: Bid to give Bush the boot

Economic Hit Man

I’ve joined a book club.

I figured that since I never have anything to do, I would sign up. :)

Anyway, our first book is Confessions of an Economic Hit Man , and I have found it fascinating. While the details in the book are not really a surprise to me (I’ve assumed this kind of thing was happening), it does seem a little hard to believe the way the story unfolds.

I will get back to you when I am finished.

Truthiness

As I wait on hold the other day, a voice keeps interrupting the music to tell me how important my call is to them. It occured to me then how insincere it seemed.Then I read A Billion People Can Be Wrong in Steve Rushin’s SI column. He points out the same thought and talks about the insincerity of many sports expressions. We now live in a world of “truthiness.”btw,While finding the link I noticed that Steve Rushin was named National Sportswriter of the Year. Congratulations, Steve.

First Post from the Mini

Got this cool mini for the kids and they love it! The power of OS X for less than $500.

Happy New Year!

I spent several hours of my 39th birthday at the Redskins v. Eagles game at the link on Sunday. I bought the tix as a Christmas present for Terri who is a huge Redskins fan. We enjoyed it very much especially since the skins made the playoffs and the Eagles failed to win a division game all season. :) I also put together a new look for this site. I wanted something a little less busy but still with all the links, photos, etc. Nothing fancy, but I like it.

RSS Readers

In a previous post I rambled on about my own personal search for the right way to manage the RSS feeds I want to read. Regular readers will remember a similar posts about link managers.

del.icio.us, furl, my linkblog etc.

I’ve been fooling around with how to post links to my linkblog and del.icio.us for a while now. I posted about this three times in July (14^th, 23^rd, 31^st) while I was playing with the best setup for me.Since then, I have toyed with a few different tools for managing this. In September, I developed a solution that I liked. I wrote a toolbar button which, when clicked, would take the info from the page I was on and post to my linkblog using a combination of builtin WordPress code and my own code from 7/14 and then post to deli.icio.us and redirect me to deli.icio.us “done” message. Aside from the occassional unescaped character, it worked beautifully for a while, and I liked it.Recently, I noticed that the links were not posting to my blog (I know some of you don’t understand why I even want this – that’s okay). After messing with it for a few minutes I decided to just do things another way. I wrote another toolbar button that would post to my linkblog using builtin WordPress functionality and popup (in a tab, of course) the del.icio.us post form with the tags populated. I copy the tags, submit to del.icio.us, paste my tags into WordPress, and submit. Quick and easy enough for me.If you use FireFox and WordPress, you can drag this link — Post Link to your toolbar.Now, edit the code as follows – replace DELICIOUSUSERNAME with your del.ici.us username and replace WORDPRESSURL with your wordpress blog URL.

Killer App

A few years ago, while on a plane to Seattle, my neighbor was reading an article in Wired magazine entitled “PowerPoint Is Evil.” I couldn’t help notice the title and once he finished with the magazine, I sttuck up a conversation with him about the article. It was an interesting discussion and my new friend let me walk away with the magazine. I am sure I sitll have it with the rest of my junk stuff somewhere which is very foolish considering I can read it anytime I’d like online.I was reminded of this recently when I heard about the role PowerPoint may have played in space shuttle disaster.

“It is easy to understand how a senior manager might read this PowerPoint slide and not realize that it addresses a life-threatening situation,” the Columbia Accident Investigation Board concluded, citing Tufte’s work. The board devoted a full page of its 2003 report to the issue, criticizing a space agency culture in which, it said, “the endemic use of PowerPoint” substituted for rigorous technical analysis.PowerPoint: The Killer App?</p> I am very sorry that people may have had to die to bring this to light, but I am glad that somebody is saying what needs to be said about this software.

Perhaps I’ve sat through too many PowerPoint presentations lately, but I think the trouble with these critics is that they don’t go far enough: The software may be as much of a mind-numbing menace to those of us who intend to remain earthbound as it is to astronauts.PowerPoint: The Killer App?</p> While checking for more info about this post, I learned that perhaps using slides to prepare a speech or presentation is not all bad. This article gives tips on good and bad use of PowerPoint for lawyers, and this one, while not about PowerPoint, explains how ‘picturing’ your speech can help give it focus. It’s an old piece, but still worth reading in my opinion. I also ran into a great story about a man who was upset because, though his presentation was a success, he didn’t get to show his slides.To make this whole episode even more rewarding, I found quite a bit of info about someideas about replacing PowerPoint. Keep in mind that these alternatives still present some of the same problems. According to Edward Tufte (the author of the Wired article), the problem is the culture of the sales pitch.