Primate Brow Flash

Latest Firefox Extension I cannon live without is Auto Pager.
Sites like Salon.com, Slate.com and nytimes.com feel the need to divide up the pages like they think I can’t concentrate for 5 pages of text all on one page. Reading 4 paragraphs and then having to click the “Next Page” button is a pain. So, here is this extension that automatically appends the next page to the bottom of the page you are on, so your reading does not get interrupted. With Auto Pager, if I load a site and then disconnect from the internet for some reason, I still have my entire article loaded in the browser for offline reading. Some, but not all, sites do have “show entire article” buttons, but this extension gives this feature to every web site.

There are many built-in sets of rules for common websites and common website software. phpBB, Google, nytimes, etc. For websites without existing rules, there is a nice wizard in Auto Pager to make new rules. The wizard allows you to click on the link that loads the next page so that it stores that and uses it every time you visit the site. It also allows you to specify which content (for example headers can be excluded) to load repeatedly. After a short learning process, most sites were very easy to create rules for. Settings allow you to control how many pages load by default.

I’m not sure yet about the privacy implications.

If you are concerned about internet privacy and security, try the NoScript Firefox extension.
NoScript’s main business is shutting down any javascript that you haven’t specifically allowed. It also provides many other features such as shutting down flash, java and unsafe web requests. One thing that scares off new NoScript users is the constant harping yellow alert bar. You can turn this off. The other thing that scares off new users: It makes the web less convenient. videos don’t play, buttons don’t work, pages reload and lose data when you enable scripting on them. These things are largely a matter of practice.
I’ve been using NoScript for about 2 years and have recently learned a bunch more about it. So here is stuff I learned from the hackademix weblog, the NoScript faq, and playing with NoScript options.

1. NoScript by default reloads all tabs affected by a new entry to the whitelist. This can cause people to lose work if, for example, they have partially filled out a form and then opened a new tab to look something up. to stop this, open a new tab, type in about:config in the url. Look up the noscript.autoreload.allTabs setting (or event the noscript.autoreload setting) and set it to false.
2. Hidden in the appearance tab is an option to display full domains. This is helpful because many ad sites have specific urls for specific websites. For example, if you want doubleclick to work on google-ads, “display full domains” allows you to whitelist googleads.g.doubleclick.net instead of allowing the entire doubleclick.net domain.
3. The “opaque” setting (options –> embeddings –> opaque embedded objects) makes embedded objects on pages opaque so that you can’t click on some invisible button by accident
4. Force secure cookies. A poorly configured site might have https but forget to mark cookies as secure. NoScript allows you to force encryption of cookies for https sites. This is off by default because it is relatively new and because some sites break if they can’t have insecure https cookies.
5. A good idea is to export your NoScript whitelist that you have built up over time so that moving to a new computer does not force you to build it again. An even better idea is to use no-script’s bookmark feature to publish your whitelist to a bookmarking system. Each instance of NoScript you use keeps track of the changes to this bookmark and updates its own whitelist accordingly.
6. Google chrome’s evolving extension framework does not yet allow for enough control to let NoScript work
7. NoScript blocks lots of stuff that are not script related. As an example, it blocks html ping elements by default.
8. Finally, some neat NoScript-specific inventions that help make you more secure:
   • ABE: Application Boundary Enforcer (ABE). Among other things, ABE prevents sites from POSTing to cross-domain resources. It strips the contents out of cross domain POST requests and turns them into GET requests.
   • ABE: If you have a specific site that needs access to LAN resources, you can publish your own ABE ruleset as a file in the root of your domain.
   • NoScript also has an invention called clear-click, which protects against click-jacking by comparing the thing you clicked with a screenshot of the page you are on. If the pictures are different, it won’t allow the click to work.

Update: Noscript also improves battery life!
Save Laptop battery with noscript

Now that most internet users have adopted ways of clearing or limiting regular cookies, many sites now use sneakier cookies that are harder to prevent and clear. The Flash plugin in your browser, which you chose to install so that you could watch YouTube videos, for instance, allows operating system access and enables web site owners to store “Flash Cookies”. This lets Flash store information in the file system outside of the browser sandbox. There is nothing built into your browser to control them. Flash cookies do not have the same constraints as normal cookies. One use of Flash Cookies is to rebuild traditional cookies after they are cleared by the user. Flash Cookies are cross browser. A cookie set in one browser can be read by another browser.

Privacy Mode in Firefox 3.5 (as well as Incognito Mode in Google’s Chrome and In Private in IE 8) do not block Flash cookies.

This information is stored on a per-site basis on your hard drive at %APP-DATA%\macromedia\flash player\#shared objects\. If you look in that folder, you will see many, many folders with names of sites you might have visited ages ago.

The existence and contents of these folders are interesting to forensics investigators, spouses, employers, and marketing professionals.

The BetterPrivacy Firefox add-on effectively wipes out the Flash cookie contents. It also wipes out the Flash cookies set by other browsers.

It does not, however, wipe out the folders containing those contents by default. You must open the Tools–> BetterPrivacy –>More Options dialog and check the “Delete Empty Cookie Folders” button. Otherwise, clues to your browsing history remain.

You can also set your flash plugin settings at the following page: http://www.macromedia.com/support/documentation/en/flashplayer/help/settings_manager03.html Many people just set their LSO storage space to zero. Note that each browser you have will maintain its own flash settings, so you would do this for each browser.

Better Privacy by default disables click pings. “ping” is an html 5 attribute that notifies a third party when you click a link. While this sounds like a huge breach of privacy, keep in mind that a website owner can and does put all kinds of tracking code to watch which links you click on. The Ping attribute in html 5 is an attempt to unify this and keep click tracking out of the normal stream of web interactions. (as read on Workbench) Anyway, it is an HTML 5 feature and I haven’t seen it used much.

Similarly, BetterPrivacy by default disables DOM Storage. Dom storage is another HTML 5 feature that provides a large space for name value pairs as client side data storage. They are different from normal cookies in that they are much bigger, they never expire, they don’t get transmitted to the server with every request, and they offer more granular control as far as scope. That is, their scope might be limited to the browser window. An application of DOM Storage might be to allow web application usage when offline, with the expectation that work would be synched up when the user was back online. This is a similar goal to, but completely different implementation from Google Gears.

Bruce Schneier made a post about flash cookies back in August. The comments on his post are really good.

I see that the Firefox profile keeps a list of the websites you have visited in the places.sqlite database and that these are cleared out when you execute Tools –> clear private data.

Download the bare bones sql-lite command line tool ( http://www.sqlite.org/download.html ) to check out what Firefox retains after you clear private data.

Navigate to %APPDATA%/mozilla/firefox/profiles to find the name of the directory (xxxxxxx.default ) where this data is stored. ( instructions for other OS are here )

and then in your sql lite directory type

sqlite3 %APPDATA%\mozilla\firefox\profiles\xxxxx.default\places.sqlite

you will then see a command line that looks like this:

sqlite>

where you can type

sqlite> .tables

to see all the tables
or

sqlite> select url from moz_places;

to see all the urls it has stored

If you go to firefox tools -> clear private data and then go back and look at all the urls again, this table will be cleared out, except for your bookmarked web sites.

As mentioned in a recent Security Now! podcast, this does not erase the information Firefox has stored about your zoom preferences! I went to one website and zoomed it into a ridiculous zoom level. Firefox 3 remembers these settings on a per-site basis. I then cleared the browser’s history and went back to find my site is still zoomed in.

I found that the zoom info is kept in a different sqlite file called content-prefs.sqlite
So, I went that sqlite file (sqlite3 “%APPDATA%”\mozilla\firefox\profiles\xxxxx.default\content-prefs.sqlite )

if you look in there and do

sqlite> "Select * from groups";

you will see a listing for each site that you visited and fiddled with the zoom settings.
so….

sqlite> delete from groups where name like '%phpsolvent%';

and that cleared my zoom settings

So, if you are truly concerned about privacy, you might want to make a special trip to clear out the groups table in content-prefs.sqlite.
I do not understand why clear private data does not wipe out this data?

You can turn the entire feature off by typing about:config in the browser’s address bar, doing a search on “zoom” and disable by by double clicking the browser.zoom.siteSpecific row.

In Firefox 3.5, site preferences including zoom preferences can be cleared from an enhanced clear private data dialog.

The unanswered question is:
Can scripting be used on one web site to check zoom level on other web pages you have visited? This concept has been used before to check the color of a link to see if you visited it before. Checking zoom level on a third party web site would obviously be a lot harder.

I got this idea from J. grossman. I extended it to list over 50 Firefox extensions. If a visitor has installed those extensions, this page can detect them.

You might be advertising the fact that you have installed hacking tools, that you have a facebook membership, that you surf using TOR to hide your identity, or even that you have installed an extension with security flaws. If you are using a public computer and wish to see which extensions are running behind the scenes, this is one way to do it.

The detection relies on javascript. There is a long discussion here on the Firefox bug list about whether to fix it. Apparently, they decided not to.

The script tries to load images associated with the extension from a chrome://… type resource. Most of the extensions I’ve tried are detectable. Some use no images at all and so this method won’t work.

There is a certain amount of path guessing involved, but most of them can be found at chrome://[extensionName]/skin/logo.png or chrome://[extensionName/skin/[extensionName].png

There are certain cases, like with greasemonkey on Mac, in which I can’t detect the extension. I suspect it is just me who can’t find it rather than any prevention in place. Also, different versions of the same extension keep their images in different places, so your version may not trigger a hit on my list.

Let me know if there are any false results.

Free yourself from the mouse and save time by learning just a few keyboard shortcuts. I’ve been trying to practice more Firefox shortcuts so this post will focus on those.

The keystrokes I use most are CTRL-t for a new tab, CTRL-L to put cursor in location bar. CTRL-e to put cursor in search box.

F7 toggles caret browsing on and off. - this lets you move the cursor around a web document and select text without using the mouse.
CTRL - U show source code (U for Under)
/ to start find-as-you-type
CTRL- TAB for switching between tabs
SHIFT-F10 for opening right-click context menu, which gives access to many more options.

Many Google applications use the J and K keys to move up and down a list. Google also uses the / key to place the cursor in a search box.
The Google Search with Keyboard “Experiment” brings keyboarding to a page of Google results. You can navigate up and down with the j and k keys and when you reach the end of a page of search results, it forwards you to the next page.
A Firefox extension called hit-a-hint powers the keyboard for web browsing and frees the user from the mouse.
This extension has a power key that makes a little number appear next to each link and control on the page. Press the number to navigate to that link or activate that control.

What should I do with all the time freed up by not using the mouse? I’m taking suggestions.

These links have many more shortcut keys.
Windows Shortcut keys
Wordpress Shortcut Keys
Firefox keyboard shortcuts
Shortcut keys for posting on blogger
Map shortcut keys to special characters across windows applications ( I wish I had known about this one when I was trying to make an umlaut in front of a bunch of people from the German School).
List of shortcuts with a comparison between browsers
Other web applications, such as flickr, have had 3rd party scripts to provide keyboard navigation

The drawback to keystrokes is that they have to be memorized or they don’t save time and every application has slightly different keystrokes. It is great that Google is bringing a more uniform set of keyboard shortcuts to their suite of applications. Having a cheatsheet near my computer helps until it gets covered with food and I throw it away. Mnemonics seem to be the most permanent way to remember them.

forums.mozillazine.org will open you up to a whole world of people who roll and smoke their own compilations of Firefox. I understand from what i read in “Firefox Hacks”, that the official Windows binary is compiled to support lots of different CPUs and therefore can’t take advantage of the enhancements of newer chips. If my system has a Dual Core 2 Gigahertz 64Bit chip and the official Firefox binary is compiled to operate on a 486, Firefox can’t fully take advantage of the power of my chip. If I installed a build compiled specifically for my processor I should see a performance enhancement. This should apply to other open source binaries like Gimp, Audacity and Eclipse.
But…

1. A third party “custom build” would be a great way to get some unsuspecting user to install something evil.
2. If there were serious enhancements to be had this way, there would be an effort to provide official binaries for many different chips, as the enhancements would make Firefox look better.
3. I think the chip in this computer is so fast that I wouldn’t notice the difference in a Firefox compiled locally.
4. If software wasn’t compiled and distributed or sold for specific chips, undermining the performance potential for all kinds of software, what incentive is there really to design new chips and for consumers to purchase them. Obviously there is an incentive and there must be big benefits for pure chip speed completely separate from the benefits of compiling specifically for your chip
5. compiling all the open source programs I use myself would be a hell of a lot of work every time there is an update.

As Steve pointed out regarding #2, maintaining lots of versions of any software would be a logistical nightmare. When it needed a security patch, the team would need to patch and compile and distribute 50 versions.

I want to see for myself. First problem is how to get Firefox to run two or more different versions on the same machine. If you download a third party version and launch it, it will just trigger your regular installed version to start. To get around this, create a new profile. (as explained here).

I also read that if I uninstall my official firefox version, I can run as many third party versions as I want and they won’t conflict as the third party versions aren’t “installs”. I’ll test this later.

So, that is how i got Bon Echo x64 2.0.0.8 running alongside my regular Firefox. Is it really faster? That will wait until the next post. I found a great page on how to measure browser speeds. I won’t link to it because the page says

“This article is around 2 years old now (although it has been kept up to date), and has been retired - posting it simply shows how long it took you to find it.”

OK, Mr. More Than Enough Readers.
The article provides everything I need to test the difference between official firefox binary and the potentially faster Custom super binary.

1. Test browser startup time with ordinary stopwatch
2. Test CSS rendering with CSS Benchmark Test
3. Test Script speed with Benchmarking tool - jsbench seems to no longer exist. I tried this other one.
4. Test Loading multiple images with an ordinary stopwatch
5. Test use of caching by navigating through search results and then doing it again - used ordinary stopwatch

He listed more tests than this, but those should be good enough to test the various builds.

I found Steve’s posts about migrating to a new computer useful, and I thought I’d copy him in detailing all the work. It is turning out to be a lot of work! Starting with just the browsers, I’ll list the extensions and settings I’ve come to depend on for safe and efficient browsing

• Find as you type for IE 7. In FireFox, if I type “/” or CTRL-F and begin typing, the browser finds words as I type. This saves a few keystrokes for me and often finds variations in spelling of the word I’m looking for. This extension makes that happen in IE as well.
• Google Keyboard Search for IE7 and FireFox: This is a change to the default search engine. Your Google results can then be browsed with the j and k keys and opened with the Enter key. It automatically jumps to the next page when you scroll off the end.
• GMail Notifier. I like having a reminder in my browser when I get mail
• Delicious toolbar links. I use these all the time. I usually just drag the delicious buttons to my bookmarks toolbar, but now I’m trying the delicious FireFox extension and I like it. It captures selected text as notes.
• Instant Library Lookup browser button. If I’m at Amazon and I see a book I’m interested in. I click the browser button and it instantly finds it at my local library. I’ve saved hundreds of dollars this way, even when I calculate in overdue fines.
• Set Firefox master password. This requires a password to access any private info in FireFox. (Tools –> Options –> Security –> Set Master Password)
• Google Dorks: This extension puts a lot of the special Google search syntax at your fingertips.
• Noscript: It is important to install this as it cuts down on possible attacks through Cross Site Scripting. I initially thought that it would be a pain in the ass to “allow” all the friendly sites to run javascript, but it is surprisingly easy. You can import allowed sites from previous installations of Firefox.
• IE7 guards against sites that “use a scripted window to ask for information” by default. This tells how to permanently ad a trusted site that won’t get blocked by this feature.
• Spelling: Firefox 2.0 has a built-in spellcheck. It is wonderful except that in my old browser I built up a big list of words that it didn’t know that I had to ask. Going into my old browser, I took the C:\Documents and Settings\\Application Data\Mozilla\Profiles\ persdict.dat and dropped into the same place on my new computer.
• IE7 has no spellcheck built in. I’ve long been using the Google toolbar for this and it is great. To import my old wordlist from old machines, I found the file “..\Google\User Dictionary.txt” and copied the word list out of there and sent it to myself thru email. No sense in teaching my spellchecker to recognize “Pr0n” “Pwnzored” “assclown” “idiotarian” “MSM”, “Suxxor”, “roxxor” and “STFU” again.
• One thing I’m trying to find is a Internet Explorer setting or extension that mimics the Firefox behavior: I click on a page of text and I can then select text with the shift key and left and right arrow keys. Any help, lazyweb?

I’m trying to find out if there are any tools out there that can recover Firefox passwords if I have Master Password set.

I downloaded a tool that is supposed to decrypt Firefox passwords, but it asks for my Master Password?

duh. why would I need this tool if I had my master password? I could just look in tools–>options–>privacy–>show passwords.

Oh well. I suppose it is good that a tool can’t recover my Firefox passwords now that I have the Master password set.

Firemaster claims to be able to brute force even the master password. I will set it to run tonight and find out the results in the morning.

Result: Firemaster can handle simple passwords, but it is much slower than John The Ripper.

Three times in as many days, I’ve wanted to recover a forgotten password. Two of these were for websites and the passwords were still stored in a browser (one in Firefox, one in Safari) and one encrypted in a htpasswd file on a server. I have learned a number of things about password storage and password recovery tools in recovering these passwords. Chiefly, it told me that anyone with access to my computer can get these passwords, especially the ones hidden by stars in the browser window like this:

Here is a list, in order of increasing difficulty, of ways to find your own passwords

1. The Firefox password was easiest to recover. I’m not happy that Firefox, by default, stores passwords in plain text. If you have Firefox, all you do is go to Tools–> options–> privacy and click the password tab. If you click the show passwords button, you will see all your saved passwords. See Securing Firefox Passwords for information about how to correct this. Firefox passwords are also stored in an encrypted form in C:\Documents and Settings\username\Application Data\Mozilla\Firefox\Profiles\xxxxxxx.default
2. for Safari on a Mac, go to applications –> Utilities –> Keychain Access, enter part of the website’s address into the search field. Double-click one of the results, click “Show password”, enter Admin password and you will see your password for that site. Beats the hell out of calling the bank.
3. stored in Internet Explorer. IE stores these auto-complete passwords encrypted in the registry. The Internet Explorer developer’s toolbar does not reveal passwords hidden by dots. The Firefox Web Developer Toolbar does. So does the DOM inspector. ( but not if you set the master password!) There are several tools available that say they recover Internet Explorer passwords. I tried the free IEPassView and found it showed me all of my passwords for autocomplete forms and any realm authentications I have done.
4. Brute Force Password Recovery Tool. This afternoon, I ran John The Ripper against a htpasswd file containing an encrypted password of mine from a year ago. It took nearly two hours, but it returned the password (8 alpha-numeric characters). It works by trying every possible sequence of characters and encrypting each one to see if it matches the string in the file. It is very simple to install and use. In fact, I just typed the following command and it was off and running:
   john htpasswd. This should destroy the myth that breaking an encrypted password or requires any special skill.
5. RainbowCrack. Haven’t tried this one yet, but it is another brute force approach. This tool generates all the possible string-encrypted string pairs and saving them in big tables known as rainbow tables. When I have an encrypted password to recover, it is just a matter of doing a search across all those huge tables for the encrypted string. This supposed to be much faster than John the Ripper.

The PEAR text_highlighter package has some options to show your code with line numbers. Chroder’s code highlighter doesn’t seem to implement this option. When I changed the code to implement it, I found that the numbers don’t line up, depending on the theme in use. I have discovered before that some wordpress themes have trouble with tables. I am testing out the web developer extension in firefox to help debug this problem. here is a screenshot of the extension in use. You can see that the extension puts nice colored boxes around all the page elements. And you can see that the numbers column is offset a little, ruining the effect of having line numbers.

**Update** solved the problem! The web developers extension is indispensable! I used it to edit the CSS of the document ‘on the fly’. That is, I can change the stylesheet in a side browser window and watch my web page change. Here is the result:

I just added the line:

.hl-gutter pre { margin: 0; padding: 0; }

to the .css file

I used to have a script in Homesite to turn table borders off and on so I could see how the layout of the web page I was making was being managed by all the nested tables. Now I have Aardvark, a firefox extension that does the equivalent for a page laid out with css. When I trigger Aardvark  and hover the mouse over an element, I see a red rectangle around the element along with its id and class and element type.  With hotkeys, I can remove the selected element, great for zapping ads, remove certain css attributes and see how this would affect the page you are viewing. Great for experimenting with css. 

This extension is not only instructive, but fun and not half as fun as its going to be when they implement certain promised features.

For "learn new stuff Friday" I wanted to see if I could get an ajax script working.  Ajax is really just a way for javascript to access data from a server and update the page without having to reload.  A simple way to start with this is to try out the Firefox Download counter

that was made by Infocraft.  It requires you to put a mirror of the feed on your server since xmlHttpRequests can only grab data from the same server.  It is all explained on his page.  So here should be a ticker for how many times the Firefox web browser has been downloaded.  It just passed 50 million:

Loading…

The feed only updates every minute, so it counts along evenly for a minute and then updates itself to correct for the real value coming from the server.

Using Google Maps and Greasemonkey, a Firefox extension that lets you program how web sites behave in your own browser, Chicago Transit rider put together an app that overlays Chicago Transit info on top of Google Maps.  ( I found this via Crooked Timber )
It fascinates me how technologies are coming together in unexpected ways. 
This brings up a question.  When I categorize this entry, do I use "mapping" or "GIS" ?   technorati and delicious have both with a strong showing by gis.  However,  not everyone knows the term GIS. Jon Udell has a good screencast about evolution of category names and tags in delicious.  If I follow his logic, delicious and technorati users will eventually settle on "mapping" or "GIS" and the tag naming system or "folksonomy" will evolve without management.
If you look at http://del.icio.us/mcgyver5  you can see that 15 other people besides me added this link to del.icio.us, but if you at the list for just this link at http://del.icio.us/url/dc136fb1ebbf8a94998023ad3783345b you can see that of those 15 people, only one other person besides me used a "gis" tag on it.