P2P Search Engines-Napster

P2P Networks: Napster Background of Napster

The Internet was originally built as a peer-to-peer system in the late 1960s to share computing resources within the US. The first hosts on ARPANET were connected together as equal peers rather than as client-server. The main users at the beginning were computing researchers who did not need protection against each other, and security break-ins were practically non-existent, making the Internet much less partitioned than it is today. Many peer-to-peer systems were widely used and still in existence today such as Usenet and DNS. In 1994 however, the structure of the Internet changed dramatically with millions of people flocking to the Net. Modem connection protocols like SLIP and PPP were widely used and applications were now targeting slow speed analog modems. Applications such as web browsers were based on a client-server protocol. The Structure of the Internet made a switch from peer-to-peer to the client-server model.

However, there are still many applications using the peer-to-peer model, one of the most successful being Napster. It all started when 19-year-old Shawn Fanning was sitting in his dorm room at Northeastern University in Boston. He was listening to his roommate complain about dead MP3 links when he came up with the idea for Napster. "I had this idea that there was a lot of material out there sitting on people′s hard drives … and I had to figure out a way to go and get it." He wanted to find an easier way for music listeners to share their favourite recordings with each other. He knew enough about Unix programming to believe such a program was achievable and dedicated all his time and effort to reaching his goal. He had to learn Windows programming as well as Unix server code. The whole concept took him over completely and he decided to drop out of college to finish his work. He moved into his uncle′s office and began his work on Napster. Once completed Napster was a huge success and became one of the fastest growing sites in history, reaching the 25 million mark in less than a year in operation. 数据挖掘实验室

Before Napster there were online recordings on the Net. Using MP3 compression format music tracks could be transferred onto disk files and then published on a website and people would download them using FTP. The one major problem with this was that up-to-date MP3 files were difficult to find. Napster solved this problem by providing constant up-to-date MP3 files in a single location that everyone knew about. Users could register with the searchable Napster network name space and find files easily through Napster servers, which had information on registered hosts and MP3 data. The servers dealt with the transfer of files between clients but didn′t actually store any of the music themselves. Napster′s network protocol created direct peer-to-peer access between clients. It is the simplicity of use that peer-to-peer provides that helped with the success of Napster and other applications that use it. 数据挖掘研究院

?/a>

Napster Architecture and How it Works

Peer-to-Peer (P2P) is a form of distributed computing that can be described as the sharing of computer resources such as files, MP3s etc and computer services by direct transfer between two computer systems. 数据挖掘研究院

Traditionally, the exchange of resources and services between computer systems is done using Client-Server techniques. A Client-Server system is one in which there is a dominant computer (the Server), that is connected to several other computers with less control (the Clients). Clients can communicate with other clients only through the Server. With P2P systems, there is no such dominant server; control is decentralized. Each node or peer on the network may act as both a client and server. Clients in a P2P network can interact freely with other clients without the intervention of a server although sometimes there is the presence of a directory server (which stores IP addresses and other information about the computers in the network) for look up purposes. The diagram below depicts the process of file transfer in a P2P network. 数据挖掘实验室

 

There are three major types of P2P network:

  数据挖掘研究院

1. Pure P2P:
   • peers act as clients and server
   • there is no central server.
   • there is no central router.
2. Hybrid P2P:
   • Has a central server that keeps information on peers and responds to requests for that information.
   • Peers are responsible for hosting the information as the central server doesn′t store files, for letting the central server know what files they want to share and for downloading its shareable resources to peers that request it.
   • Route terminals are used addresses, which are referenced by a set of indices to obtain an absolute address.
3. Mixed P2P:
   • Has both pure and hybrid characteristics.

Napster is an example of a hybrid P2P system. Napster has a centralised directory (actually several) that describes how files reside in Napster and each host registers with this directory when they join the network. The centralised directories therefore have the IP addresses, the names of the files the hosts want to share and other data stored about each computer system connected to it.

How does Napster work?

1. Each user must have Napster software in order to partake in file transfers. The user runs the Napster program. Once executed, this program checks for an Internet connection.
2. If an Internet connection is detected, another connection between the user′s computer and one of Napster′s Central Servers will be established. This connection is made possible by the Napster file-sharing software.
3. The Napster Central Server keeps a directory of all client computers connected to it and stores information on them as described above.
4. If a user wants a certain file, they place a request to the Napster Centralised Server that it′s connected to.
5. The Napster Server looks up its directory to see if it has any matches for the user′s request.
6. The Server then sends the user a list of all that matches (if any) it as found including the corresponding, IP address, user name, file size, ping number, bit rate etc.
7. The user chooses the file it wishes to download from the list of matches and tries to establish a direct connection with the computer upon which the desired file resides. It tries to make this connection by sending a message to the client computer indicating their own IP address and the file name they want to download from the client.
8. If a connection is made, the client computer where the desired file resides is now considered the host. The host now transfers the file to the user.
9. The host computer breaks the connection with the user computer when downloading is complete.

How does Napster search for and trade files? Client Server Protocol

Napster searches for files using the client/server protocol. The client/server protocol uses TCP/IP communication protocol. The server maintains a ′master list′ of all computers connected to it and it searches for files by looking up its directory as described in some of the steps above. Client/Client Protocol 数据挖掘研究院

Napster allows file transfer that is independent of the server. It occurs between two client computers. There are four transfer modes: upload, download, firewall upload and firewall download.

数据挖掘研究院

?/a>

Legal Issues surrounding Napster

After it′s creation in May 1999 by founder Shawn Fanning, many record companies realised that the threat Napster posed to its potential earnings was immense and hence worth pursuing in a legal manner. This court case involved the Recording Industry Association of America (RIAA), which includes such music industry giants as AOL Time Warner′s Warner Music, BMG, EMI and Sony Music among others, suing Napster over breach of copyright law, saying that it had illegally founded a business based on the use of copyrighted material that they did not have privilege to distribute. They believed that the whole file-sharing movement would cost the music industry millions in falling sales and unpaid royalties, while Napster argued that it was merely providing a service and it was the users who were doing the distributing. The following is an account of the legal proceedings involving both parties, which started just seven months after Napster was founded.

Initially, the RIAA had contacted Napster several times requesting (including in writing) the removal of copyrighted material from their system and only after they claimed to have received no response did they decide to file a lawsuit seeking an injunction. Contrary to this, Napster claimed to have responded (in writing), seeking to arrange some sort of settlement, which would have suited both parties. At this point, they also added a disclaimer on their system, saying that playing, transmitting, copying and creating any music related files may be impinging on federal law and it was up to the user whether they were to adhere to this or not.

数据挖掘实验室

In December 1999, the RIAA filed their lawsuit at the Ninth US District Court in Northern California (in San Francisco) against Napster, saying they broke both federal and state laws through "contributory and vicarious copyright infringement" and that they wished to have the service shut down permanently. Obviously, providing a list of music files that existed available for anyone with Internet access to view and use is hardly breaching copyright law - this lawsuit meant the court would have to decide the closeness of Napster′s relationship with the users in facilitating such practices. The reason that these companies were targeting Napster (and why they felt they had a case) was the fact that Napster used a third party directory service to facilitate its users, while clones such as Gnutella merely set-up a direct P2P link between two hosts.

On July 26 2000, Judge Patel of San Francisco ordered an injunction on Napster to halt its service, siding with the record companies, while still awarding Napster the chance to appeal. Napster argued that personal copying of music is protected by federal law under the Audio Home Recording Act of 1992, while the judge declared that computers were not classified as home recording devices. They claimed that the injunction would put them out of business - in contrast, record companies said Napster was costing them over $300 million/year in lost sales. They also (somewhat cheekily) claimed that it did not know that copyright infringement was taking place (i.e. "all this MP3 sharing is nothing to do with us!") - Judge Patel was not impressed!

Two days later, Napster successfully appealed the injunction and were allowed to remain in operation, at least for the time being. The case now needed a full trial and it was up to the RIAA to pursue it, either as far as the Ninth US District Court or all the way to the US Supreme Court. They were very upset at the success of the appeal but expected the higher courts to rule in their favour once it looked at the case in detail. In response to their success, jubilant Napster spokespeople urged their users to continue what they called their ′Buycott′, i.e. continue to buy published music by artists that supported free trade of digital music.

In early October of the same year, both parties were ordered to present each of their arguments in front of the District Court. At this point, the record companies realised that a complete shutdown of Napster was unlikely and hence requested that their copyrighted material be removed/blocked from the system and if not that Napster would be charged with copyright infringement. On the other hand, Napster were hoping to come to an arrangement with the music industry by hoping to establish a fee paying service, where users paid a nominal monthly fee, most of which would be forwarded to the record companies who owned the copyright for the downloaded material. This idea was not embraced by the RIAA, who still wished to have the matter settled in the courts. 数据挖掘研究院

However, it is worth mentioning that by the end of this month, BMG (a member of the RIAA) announced that it was to work with Napster on a similar system to that suggested and in the end withdrew from the legal proceedings. This became a reality at the end of January 2001.

数据挖掘研究院

Despite what was happening with BMG, the other companies were still pursuing their case and seemed to have obtained what the desired at a ruling on February 12, 2001. The judge ruled that companies had to provide Napster with details (i.e. title, artist name, names of files on Napster relating to these works, certification that the companies control rights were infringed) of the files they wanted removed from the system and it was Napster′s responsibility to carry it out. Both parties also had to undertake reasonable measures to remove files with name variations of such files, in order to prevent users using e.g. shortened or misspelled file names. Once this was done, Napster would have 72 hours to remove the files from their system. 数据挖掘研究院

By March 28, there was still conflict between those involved. On one side, Napster were saying that they had removed over 250,000 unique songs with over 1.6 million different filenames, while on the other hand, record companies claimed to have given them 8 million file names, covering over 600,000 different works. The two main problems were with misspelled filenames, which could not easily be related to a particular piece of material and the fact that many of the song names given to Napster came with no associated filename. 数据挖掘研究院

Eventually, and somewhat ironically, Napster ended up coming to an agreement with AOL Time Warner and EMI as well as several independent record labels in Britain and Europe, similar to the existing one with BMG, where they would receive payment for files downloaded. Having being ordered to remain offline until all copyrighted material had been removed, Napster spiraled into bankruptcy, and with a court ruling blocking the sale of Napster to Bertellsman AG, the last hopes of reviving the service disappeared. 数据挖掘研究院

?/a>

The legacy of Napster: Online warfare on P2P networks

Ever since the court ruling in July 2000 ordering the shutting down of Napster′s file swapping service the question has been raised as to how to enforce such a decision, not just against Napster but against the plethora of similar services which have appeared in the wake of the company′s bankruptcy. Here we explore how music and film labels since Napster are fighting file swapping with "hacking" techniques 数据挖掘实验室

Systems such as Gnutella do not have any central directory server as Napster did. To deal with these, something is needed which attacks the overall structure of P2P networks. The new techniques being considered are technologically based, and in many cases music labels are considering attacking these services using "hacking techniques". 数据挖掘研究院

During the Napster legal battle, the RIAA (Recording Industry Association of America) included among the defendants several universities whose students were hosting file servers using the university network. Many of these subsequently agreed to block all traffic relating to Napster in order to avoid further legal proceedings. Similarly, some ISPs have started to discourage or ban their subscribers from serving files from their machines. [http://news.dmusic.com/print/5683] As part of the Digital Millennium Copyright Act [http://lcweb.loc.gov/copyright/legislation/dmca.pdf], ISPs can be forced to identify suspected infringers of copyright. This process is inevitably slow, being complicated by legal wrangling, and anti-piracy techniques are now being used that do not require court intervention. 数据挖掘实验室

A method routinely employed at present is known as "spoofing", and involves flooding repositories of copyrighted material with fake, corrupt or otherwise undesirable files. The objective here is that users will find the services extremely unreliable and hence be discouraged from simply browsing and downloading files since there would be a high probability of getting a file containing nothing but 3 minutes of silence, for example. Artists such as Eminem have used this immediately preceding the release of certain recent albums, and several P2P networks are reporting co-ordinated spoofing attacks on their networks from sources using narrow bands of IP addresses, presumably corporate networks.

数据挖掘研究院

But while these types of attacks may be irritating for P2P network users, many networks are adapting to these techniques, and the forthcoming version of Morpheus will contain a rating system whereby users can rate the authenticity of files, thereby allowing others to avoid downloading a file which has been marked as bogus. This is leading the music industry to consider other approaches to fighting piracy, including those normally associated with hackers, such as denial-of -service attacks and domain name hijacking. 数据挖掘研究院

A denial-of-service attack involves bombarding a site or machine on the net with requests such that it becomes overloaded and unable to serve real users. Multiple very slow downloads can be started which effectively clog the entire bandwidth of a target machine. Domain name hijacking is generally accomplished by changing DNS information so that traffic destined for a particular site gets redirected to a location chosen by the hijacker. As well as impairing P2P sites, the traffic could be analysed, potentially leading to more targets for attack and effectively "infiltrating" the system. A more extreme example is the suggestion that purpose designed worms and viruses could be used to attack and disable machines running P2P software. 数据挖掘研究院

The important difference between this form of "hacking" and the approaches mentioned earlier is that hacking of any form is, at present, illegal. "At present" is the important phrase here, because a proposed bill written by US Congressman Howard Berman would provide a degree of legal immunity to copyright holders hacking in pursuit of copyright violations. The exact techniques, which would be permitted, is to be kept secret and only revealed to the US Attorney General, however intentional deletion of files would be forbidden. The new rules would make it quite difficult for those whose systems have been damaged by such an attack to claim any form of compensation, since they would require the Attorney General′s permission before filing any lawsuit, and they could only claim if the cost of the damage was greater than $250.

Whilst some may argue that it is appropriate to allow these organisations to use the tools they require to protect their business, the bill is seen by others as going very much too far, and being a license to distribute viruses or generally cause whatever type of mayhem Hollywood sees fit to impair P2P. Berman says that: 数据挖掘实验室

"Congress should free copyright creators and owners to develop and deploy technological tools for addressing P2P piracy" [http://www.house.gov/berman/p2p062502.html] 数据挖掘实验室

From a legal standpoint it would seem that copyright holders have the right to attempt to disable or block P2P networks, however their options are presently limited by what is conceived as being malicious use of computers. Providing them with an all purpose get-out-of-jail-free card may certainly help to deter those who would flaunt copyright law, however there is a grave risk that they could cause accidental damage to a completely innocent Internet user who would have few rights as concerns compensation. Steve Griffin of StreamCast networks says: "Even law enforcement officers are not above the law; copyright owners certainly should not be" [http://news.com.com/2010-1078-942330.html]

数据挖掘研究院

It is easy to forget that there are many legitimate uses for P2P systems, such that giving someone permission to sabotage them at will could seriously impact the development of what has undoubtedly shown itself to be a useful technology. Clumsy, ill-conceived and damaging hacking attacks could potentially be carried out with little way to extract any form of recompense from those responsible. It is in essence setting the stage for a high-tech battle between some of the worlds largest companies and a vast file-sharing community, who make more than 3 billion P2P downloads per month.

数据挖掘研究院

What direction will be taken from here is uncertain. Berman′s legislation was accepted by the US congress in July [http://www.house.gov/berman/pr072502.htm], and he has since come under heavy attack in the media, from the Electronic Frontier Foundation [http://www.eff.org/IP/P2P/20020802_eff_berman_p2p_bill.html] and others [http://www.wired.com/news/politics/0,1283,54153,00.html], despite his claims that the bill does not allow for unrestrained interference with P2P networks [http://www.house.gov/berman/remarks092602.htm]. At this stage it is difficult to judge the effects of the new laws, however it is clear that the recording industry′s victories against Napster in the courts are only the beginning of a major conflict which will take place, most likely not in the courtroom this time, but online.

?/a>

A Napster Timeline

• May 1999: Napster online music service is founded by Shawn Fanning and Sean Parker. Napster, a peer-to-peer file (P2P) sharing service was developed primarily to copy and swap unencrypted music for free, such as MP3s (a format which compresses the recordings into small, easily shared files, while retaining a high quality of sound).
• December 1999: The RIAA file a suit against Napster, on behalf of five major U.S. recording labels, accusing the service of copyright infringement and encouraging the illegal copying and distribution of music. They seek an injunction against Napster for damages and for lost revenue from thousands of songs it claimed were lifted through Napster′s program. To Napster′s advantage though, earlier in the year the 9th U.S. Circuit Court of Appeals ruled that Diamond Multimedia′s portable MP3 player (introduced in 1998) did not violate the Audio Home Recording Act of 1992, which prohibits devices that make copies from digital music recordings. It was decided that the copies were produced from computer hard drives, not digital music recordings.
• January 2000: Napster′s popularity grinds many academic computer networks to a halt, as college students in particular embrace the online music swapping with their access to free large bandwidth making the rapid up/ downloading of recordings far more attainable. This prompts some universities to ban the service.
• April 2000: Metallica and Dr. Dre sue Napster for violating state and federal copyright laws and racketeering. They also attack the University of Southern California, Yale University and Indiana University for refusing to censor students′ Web access to Napster.
• May 2000: Finally musicians come out on Napster′s side, Rapper Chuck D, representing pro-Napster groups, tells the committee of House Small Business that the Web poses "a unique opportunity for consumers and artists". Rap-metal band Limp Bizkit, along with Offspring and Courtney Love, join him in speaking out on Napster′s behalf.
• Napster′s contention that it was only a "conduit" for information, it′s defense against the RIAA suit of copyright infringement, is rejected by Judge Patel, thereby forcing the motion into the courtroom. Napster goes on to block the passwords of 335,000 of it′s users apparently involved in swapping Metallica songs, in observing the "notice and take down" section of the Digital Millennium Copyright Act of 1998.
• June 2000: The RIAA files a motion to have all major-label content removed from Napster′s system. Napster dispute the charge of being a "haven" of piracy with the claim that it is simply an ISP. They hire David Boies, lead attorney in the U.S. Department of Justice challenge of Microsoft, to aid them in their defense against charges filed by the RIAA, Metallica and Dr. Dre.
• July 26, 2000: Judge Marilyn Hall Patel issues a preliminary injunction to shut down Napster, saying the Web site infringes copyrights held by record labels and composers. She claimed it was clear that Napster users were not simply engaging in swapping their personal favorite songs, but that service was encouraging "wholesale infringing" against the music industry. Patel burdens Napster with full responsibility for removing all of the plaintiffs′ copyrighted music from its service. To this, Napster argue the technological impossibility of removing the works in question in vain. Two days later, Napster′s Hank Barry and his legal team (including Boies) succeed in having the injunction stayed, ruling that substantial questions were raised about the merits and form of Patel′s injunction.
• October 2000: Further attempts are made by the RIAA, supported by 21 other groups, to have the ban forbidding copyrighted songs appearing on Napster′s website reimposed. This would effectively spell the end of the service (which has over 30 million users at this point!). Meanwhile, attorneys representing Metallica, Dr. Dre et al, are haranguing universities to restrict students access to Napster.
• In a surprise announcement on October 31, Shawn Fanning and Hank Barry (now CEO of Napster) reveal a partnership deal with Bertelsmann AG, a German media and publishing giant and one of their main opponents to this stage, to create a subscription-based membership, thereby guaranteeing payment to the artists. In return for a stake in Napster, BMG agree top withdraw their suit.
• December 2000: This does not end their in-court woes though, as weeks after Napster and Bertelsmann form an alliance, German law-enforcement authorities charge them with enabling the spread of right-wing extremist music. Furthermore, U.S. Senator Orrin Hatch, Utah Republican, comes to the forefront in the congressional battle over online entertainment services including Napster.
• January 2001: The Dave Matthews Band becomes the first major-label band to officially distribute a song to the file-sharing program with the labels′ permission. A deal with Germany′s Edel Music AG gives Napster access to one of the world′s largest independent music labels. TVT Records drops its copyright infringement lawsuit against Napster. With things looking positive, Bertelsmann announces that membership fee for users of Napster will come into effect in the summer of 2001.
• February 2001: The 9th U.S. Circuit Court of Appeals rules that Napster was fully aware that it′s users were trading illegal music. The highly damaging memo by Napster′s co-founder Sean Parker where he claimed the company needed to remain ignorant about the "real names" of the users because "they are exchanging pirated music", was used as evidence. The court found that Napster was involved in "contributory and vicarious infringement," and had full knowledge that it was allowing its users to violate copyright laws. The judges agree that the recording companies have proven "Napster users infringe at least two of the copyright holders′ exclusive rights: the rights of reproduction and distribution." Napster′s offer of US$1 billion to the RIAA for the rights to their music is decried and rejected by the companies.
• March 2001: Napster begin the process of blocking copyrighted music from their site, with much haggling in the courts, claiming not enough information is being provided by the records companies… a number of court visits results in the RIAA supplying Napster with a list of 135,000 copyrighted songs that the labels want removed from their service. The data on the list of songs to be removed includes the names of artists, the songs′ titles and albums on which the songs appear. Napster users begin to scramble the letters in file names to make filtering and removing files much more difficult. Emusic.com Inc. and the producers of the Grammy Awards sue Napster Inc.
• April 2001: Judge Patel describes Napster as an out-of-control "monster" that should be shut down entirely, and claims Napsters efforts to comply with her rulings were "disgraceful". With scrambling and intentional typos, Napster users are making it extremely difficult to control copyrighted material. Napster begins using a variety of tracking and ′acoustic fingerprinting′ techniques to find and remove files.
• June 2001: Napster′s upcoming subscription service is boosted with the signing up of 150 British independent record labels.
• July 2001: Judge Patel orders Napster offline until such time as it is proven to be entirely free of illegal music. Metallica and Dr. Dre finally settle their lawsuits with Napster and Hank Barry is replaced as CEO by former BMG executive Konrad Hilbers.
• October 2001: An offline Napster seals a licensing deal with the Harry Fox Agency (a division of the National Music Publishers Association). It is planned to settle pending suits brought against Napster by multiple publishing companies. The subscription service is set to launch before the end of the year.
• January 2002: Sees Napster still offline, while pending lawsuits are put on hold as settlements are sought.
• March 2002: Napster is ordered to remain offline and financial difficulties and board member conflicts dog their path.
• June 2002: The cash-poor company file for Chapter 11 bankruptcy in an effort to protect themselves from creditors as Bertelsmann AG prepare for a complete buy-out.
• September 2002: After much objection, Judge Peter J. Walsh blocks the sale of Napster to Bertelsmann. The "conflicting interests" of CEO Konrad Hilbers is particularly noted. This court ruling destroys any shred of hope of Napster reclaiming its′ once thriving place online. Most of the staff including Shawn Fanning find themselves unemployed as plans for a Chapter 7 bankruptcy liquidation of the company′s assets begins.

?/a> 数据挖掘研究院

Bibliography

  数据挖掘研究院

http://personal.ansir.com/wms/fanning.htm

数据挖掘实验室

http://www.oreilly.com/catalog/peertopeer/chapter/ch01.html

http://compnetworking.about.com/library/weekly/aa062500a.htm 数据挖掘研究院

http://rr.sans.org/policy/post_napster.php 数据挖掘研究院

http://webopedia.internet.com/ComputerScience/ClientServerComputing/peertopeerarchitecture.html 数据挖掘研究院

www.howstuffworks.com 数据挖掘实验室

http://wiki.cs.uiuc.edu/cs427 数据挖掘研究院

http://www.cnn.com/SPECIALS/2001/napster

http://napster.findlaw.com/ 数据挖掘研究院

http://www.idg.net/english/crd_napster_497766.html

http://www.idg.net/idgns/2001/04/02/NapsterTimeline.shtml

数据挖掘研究院

http://grammy.aol.com/features/0130_naptimeline.html
数据挖掘实验室

http://www.personal.psu.edu/users/j/i/jid102/timeline.html