Robots (or spiders, or crawlers) are little computer programs that search engines use to scan and index websites. Robots.txt is a little file placed on webservers to tell search engines what they should and shouldn’t index. The Internet Archive isn’t a search engine, but has historically obeyed exclusion requests from robots.txt files. But it’s changing its mind, because robots.txt is almost always crafted with search engines in mind and rarely reflects the intentions of domain owners when it comes to archiving. Over time we have observed that the robots.txt files that are geared toward search engine crawlers do not necessarily serve our archival purposes. Internet Archive’s goal is to create complete “snapshots” of web pages, including the duplicate content and the large versions of files. We have also seen an upsurge of the use of robots.txt files to remove entire domains from search engines when they transition from a live web site into a parked domain, which has historically also removed the entire domain from view in the Wayback Machine. In other words, a site goes out of business and then the parked domain is “blocked” from search engines and no one can look at the history of that site in the Wayback Machine anymore. We receive inquiries and complaints on these “disappeared” sites almost daily. A few months ago we stopped referring to robots.txt files on U.S. government and military web sites for both crawling and displaying web pages (though we respond to removal requests sent to email@example.com). As we have moved towards broader access it has not caused problems, which we take as a good sign. We are now looking to do this more broadly. An excellent decision. To be clear, they’re ignoring robots.txt even if you explicitly identify and disallow the Internet Archive. It’s a splendid remember that nothing published on the web is ever meaningfully private, and will always go on your permanent record.
Archive for the ‘reader’ Category
Today, Munich-based Lilium Aviation conducted the first test flight of its all-electric, two-seater, vertical take-off and landing (VTOL) prototype. “In a video provided by the Munich-based startup, the aircraft can be seen taking off vertically like a helicopter, and then accelerating into forward flight using wing-borne lift, ” reports The Verge. From the report: The craft is powered by 36 separate jet engines mounted on its 10-meter long wings via 12 movable flaps. At take-off, the flaps are pointed downwards to provide vertical lift. And once airborne, the flaps gradually tilt into a horizontal position, providing forward thrust. During the tests, the jet was piloted remotely, but its operators say their first manned flight is close-at-hand. And Lilium claims that its electric battery “consumes around 90 percent less energy than drone-style aircraft, ” enabling the aircraft to achieve a range of 300 kilometers (183 miles) with a maximum cruising speed of 300 kph (183 mph). “It’s the same battery that you can find in any Tesla, ” Nathen told The Verge. “The concept is that we are lifting with our wings as soon as we progress into the air with velocity, which makes our airplane very efficient. Compared to other flights, we have extremely low power consumption.” The plan is to eventually build a 5-passenger version of the jet. Read more of this story at Slashdot.
Enlarge / Pictured: Probably an editor who peer-reviewed stuff for Tumor Biology . (credit: flickr user: 派脆客 Lee ) The journal Tumor Biology is retracting 107 research papers after discovering that the authors faked the peer review process. This isn’t the journal’s first rodeo. Late last year, 58 papers were retracted from seven different journals— 25 came from Tumor Biology for the same reason. It’s possible to fake peer review because authors are often asked to suggest potential reviewers for their own papers. This is done because research subjects are often blindingly niche; a researcher working in a sub-sub-field may be more aware than the journal editor of who is best-placed to assess the work. But some journals go further and request, or allow, authors to submit the contact details of these potential reviewers. If the editor isn’t aware of the potential for a scam, they then merrily send the requests for review out to fake e-mail addresses, often using the names of actual researchers. And at the other end of the fake e-mail address is someone who’s in on the game and happy to send in a friendly review. Read 6 remaining paragraphs | Comments
Images of Seleznev with stacks of cash were found on his laptop following his 2014 arrest in the Maldives. (credit: Department of Justice ) Russian hacker Roman Seleznev was sentenced to 27 years in prison today. He was convicted of causing more than $169 million in damage by hacking into point-of-sale computers. Seleznev, aka “Track2,” would hack into computers belonging to both small businesses and large financial institutions, according to prosecutors. He was arrested in the Maldives in 2014 with a laptop that had more than 1.7 million credit card numbers. After an August 2016 trial, Seleznev was convicted on 38 counts, including wire fraud, intentional damage to a protected computer, and aggravated identity theft. The sentence is quite close to the 30 years that the government asked for. Prosecutors said Seleznev deserved the harsh sentence because he was “a pioneer” who helped grow the market for stolen credit card data and because he “became one of the most revered point-of-sale hackers in the criminal underworld.” Read 6 remaining paragraphs | Comments
Gulping down an artificially sweetened beverage not only may be associated with health risks for your body, but also possibly your brain, a new study suggests. From a report: Artificially sweetened drinks, such as diet sodas, were tied to a higher risk of stroke and dementia in the study, which published in the American Heart Association’s journal Stroke on Thursday. The study sheds light only on an association, as the researchers were unable to determine an actual cause-and-effect relationship between sipping artificially sweetened drinks and an increased risk for stroke and dementia. Therefore, some experts caution that the findings should be interpreted carefully. No connection was found between those health risks and other sugary beverages, such as sugar-sweetened sodas, fruit juice and fruit drinks. Read more of this story at Slashdot.
On Thursday, the Center for Constitutional Rights challenged the NYPD’s body camera polici es , asking a judge to block the city’s forthcoming pilot program, which is slated to outfit 1, 000 officers with body cameras as early as next week. The cameras were supposed to be a step forward for police accountability and… Read more…
uTorrent is the most popular Bittorrent client in the world, but it’s clearly getting a bit long in the tooth. You can expect some big changes soon, though. TorrentFreak reports that the app will eventually run in your web browser, based on comments from BitTorrent creator Bram Cohen in an interview with the Steal This Show podcast. The move will allow uTorrent the offer better streaming support — something the current client has always struggled with — and it’ll also give its developers access to more modern technology to add even more features. And, surprisingly enough, you’ll likely see elements from the company’s defunct Maelstrom browser in the new client too. uTorrent will take its time before forcing the client on users, though. “We know people have been using uTorrent for a very long time and love it, ” Cohen said. “So we’re very, very sensitive to that and gonna be sure to make sure that people feel that it’s an upgrade that’s happening. Not that we’ve just destroyed the experience.” Via: TorrentFreak
Do the new McDonald’s uniforms remind you of anything ? If you answered “every dystopian sci-fi movie ever, ” you’re correct. To me, they invoke a very Logan’s Run future. But mandatory gray-on-gray with a dash of black is pretty much universally recognized as the standard uniform for bleakest of futures. Read more…