that web page. And not all votes are equal. Votes cast by web pages that are themselves important are weighted more heavily than links from web pages that have low PageRanks (because no one else links to them).
Google found that web pages with higher PageRanks were more likely to contain the information that users are actually seeking. And itâs very hard for users to manipulate their own PageRank. Merely creating a bunch of new web pages that link to your home page wonât work because only links from web pages that themselves have reasonably high PageRanks will have an impact. And itâs not so easy to create web pages that other sites will actually link to.
The PageRank system is a form of what webheads call âsocial network analysis.â Itâs a good kind of guilt by association. Social network analysis can also be used as a forensic tool by law enforcement to help identify actual bad guys.
Iâve used this kind of data mining myself.
A couple of years ago, my cell phone was stolen. I hopped on the Internet and downloaded the record of telephone calls that were made both to and from my phone. This is where network analysis came into play. The thief made more than a hundred calls before my service was cut off. Yet most of the calls were to or from just a few phone numbers. The thief made more than thirty calls to one phone number, and that phone number had called into the phone several times as well. When I called that number, a voice mailbox told me that Iâd reached Jessicaâs cell phone. The third most frequent number connected me with Jessicaâs mother (who was rather distraught to learn that her daughter had been calling a stolen phone).
Not all the numbers were helpful. The thief had called a local weather recording a bunch of times. By the fifth call, however, I found someone who said heâd help me get my phone back. And he did. A few hours later, he handed it back to me at a McDonaldâs parking lot. Just knowing the telephone numbers that a bad guy calls can help you figure out who the bad guy is. In fact, cell phone records were used in just this way to finger the two men who killed Michael Jordanâs father.
This kind of network analysis is also behind one of our nationâs efforts to smoke out terrorists.
USA Today
reported that the National Security Agency has been amassing a database with the records of two trillion telephone calls since 2001. Weâre talking thousands of terabytes of information. By finding out who âpeople of interestâ are calling, the NSA may be able to identify the players in a terrorist network and the structure of the network itself.
Just like I used the pattern of phone records to identify the bad guy who stole my phone, Valdis Krebs used network analysis of public information to show that all nineteen of the 9/11 hijackers were within two email or phone call connections to two al-Qaeda members whom the CIA already knew about before the attack. Of course, itâs a lot easier to see patterns after the fact, but just knowing a probable bad guy may be enough to put statistical investigators on the right track.
The 64,000-terabyte question is whether itâs possible to start with just a single suspect and credibly identify a prospective conspiracy based on an analysis of social network patterns. The Pentagon is understandably not telling whether its data-mining contractorsâwhich include our friend Teradataâhave succeeded or not. Still, my own experience as a forensic economist working to smoke out criminal fraud makes me more sanguine that Super Crunching may prospectively contribute to homeland security.
Looking for Magic Numbers
A few years ago, Peter Pope, who was then the inspector general of the New York City School Construction Authority, called me and asked for help. The Construction Authority was spending about a billion dollars a year in a ten-year plan to renovate New York City schools. Many of
Google found that web pages with higher PageRanks were more likely to contain the information that users are actually seeking. And itâs very hard for users to manipulate their own PageRank. Merely creating a bunch of new web pages that link to your home page wonât work because only links from web pages that themselves have reasonably high PageRanks will have an impact. And itâs not so easy to create web pages that other sites will actually link to.
The PageRank system is a form of what webheads call âsocial network analysis.â Itâs a good kind of guilt by association. Social network analysis can also be used as a forensic tool by law enforcement to help identify actual bad guys.
Iâve used this kind of data mining myself.
A couple of years ago, my cell phone was stolen. I hopped on the Internet and downloaded the record of telephone calls that were made both to and from my phone. This is where network analysis came into play. The thief made more than a hundred calls before my service was cut off. Yet most of the calls were to or from just a few phone numbers. The thief made more than thirty calls to one phone number, and that phone number had called into the phone several times as well. When I called that number, a voice mailbox told me that Iâd reached Jessicaâs cell phone. The third most frequent number connected me with Jessicaâs mother (who was rather distraught to learn that her daughter had been calling a stolen phone).
Not all the numbers were helpful. The thief had called a local weather recording a bunch of times. By the fifth call, however, I found someone who said heâd help me get my phone back. And he did. A few hours later, he handed it back to me at a McDonaldâs parking lot. Just knowing the telephone numbers that a bad guy calls can help you figure out who the bad guy is. In fact, cell phone records were used in just this way to finger the two men who killed Michael Jordanâs father.
This kind of network analysis is also behind one of our nationâs efforts to smoke out terrorists.
USA Today
reported that the National Security Agency has been amassing a database with the records of two trillion telephone calls since 2001. Weâre talking thousands of terabytes of information. By finding out who âpeople of interestâ are calling, the NSA may be able to identify the players in a terrorist network and the structure of the network itself.
Just like I used the pattern of phone records to identify the bad guy who stole my phone, Valdis Krebs used network analysis of public information to show that all nineteen of the 9/11 hijackers were within two email or phone call connections to two al-Qaeda members whom the CIA already knew about before the attack. Of course, itâs a lot easier to see patterns after the fact, but just knowing a probable bad guy may be enough to put statistical investigators on the right track.
The 64,000-terabyte question is whether itâs possible to start with just a single suspect and credibly identify a prospective conspiracy based on an analysis of social network patterns. The Pentagon is understandably not telling whether its data-mining contractorsâwhich include our friend Teradataâhave succeeded or not. Still, my own experience as a forensic economist working to smoke out criminal fraud makes me more sanguine that Super Crunching may prospectively contribute to homeland security.
Looking for Magic Numbers
A few years ago, Peter Pope, who was then the inspector general of the New York City School Construction Authority, called me and asked for help. The Construction Authority was spending about a billion dollars a year in a ten-year plan to renovate New York City schools. Many of
Similar Books
