2012年5月7日星期一

weinre - Home

weinre - Home

weinre - Home

Home
- User Interface
- Installing
- Running
- Multi-User
- Security
- Building
- ChangeLog
- License

 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

weinre is WEb INspector REmote.
Pronounced like the word "winery". Or maybe like the word "weiner". Who knows,
really.

weinre is a debugger for web pages, like
FireBug (for FireFox)
and
Web Inspector (for WebKit-based browsers), except it's designed
to work remotely, and in particular, to allow you debug web pages
on a mobile device such as a phone
.

Interesting places to visit:

videos at YouTube:

http://www.youtube.com/results?search_query=weinre

this documentation:

http://people.apache.org/~pmuellr/weinre/ [temporary]

unofficial binary packages:

http://people.apache.org/~pmuellr/weinre/ [temporary]

issues:

https://issues.apache.org/jira/gobble-de-gook/...

create a new issue:bug
or
new feature

 

cloneable source at Apache:

https://git-wip-us.apache.org/repos/asf/incubator-cordova-weinre.git

Apache repo cloned at GitHub:

https://github.com/apache/incubator-cordova-weinre

mailing lists at Apache:

http://incubator.apache.org/cordova/#mailing-list (use callback-dev)

discussion at Google Group:

http://groups.google.com/group/weinre

More information on weinre's big brothers, Web Inspector / Google Chrome
Developer Tools, is available at
Google's Chrome Dev Tools pages
and
Apple's Web Inspector pages.

If you aren't familiar with FireBug or Web Inspector,
weinre isn't going to make too much sense to you.
weinre reuses the user interface code from the
Web Inspector project at WebKit,
so if you've used Safari's Web Inspector or Chrome's Developer Tools,
weinre will be very familiar. If you're not
familiar with Web Inspector, the links above to Google's and Apple's documentation
will be useful.

Here's an example session using weinre:

Two screen captures are shown above. On the left is the
weinre client application, displaying debug
information.
On the right is the target web page, running in Mobile Safari in the iOS Simulator.

In normal usage, you will be running the client application in a browser
on your desktop/laptop, and running a target web page on your mobile device.

In the session above, the debugger is display the Elements panel, with the
embedded console expanded in the bottom third of the window. In the console
area, we've entered two statements:

document.body.style.backgroundColor
document.body.style.backgroundColor = "pink"

The result of running the first statement shows the value of the body's background style is not set.
The second statement sets the body's background to pink. That horrifying pink
background is then rendered in the target's window.

In addition, an h1 element is selected in the upper third of the
Elements panel, which causes the side panels on the right to be filled in with
relevant information. In this case, the matched CSS rules for that h1
element are being shown.

If you are familiar with WebKit's Web Inspector,
a partial list of differences between it and weinre
are listed below:

  • weinre does not make use of any 'native' code,
    it's all JavaScript (and currently Java code for the 'server').

     

  • Because weinre doesn't use 'native' code,
    it's functionality is limited. For instance, source level debug of JavaScript
    is not possible
    easy.

     

  • Because weinre doesn't use 'native' code,
    the debug target code will run on browsers
    without specialized debug support. For instance, a browser running on your phone.

     

  • weinre supports 'remote' interaction,
    so you can run the debugger user interface on one
    machine and can debug a web page running
    on another machine. For instance, debug a web page displayed on your phone from your
    laptop.

     

  • Because weinre supports 'remote' interaction,
    multiple debug clients can be debugging the same debug target at the same time.

     

  • One thing not different from Web Inspector is that the debug client
    user interface only runs on WebKit-based browsers.

     

Supported Libraries and Platforms

Generally version numbers listed here are the earliest
versions of the relevant thing that have been tested.
Later versions of those things are also hopefully supported,
but let us know.

Libraries not supported

  • versions of Prototype.js before version 1.7 are not supported,
    as they do not support the JSON.stringify() API correctly. You
    will get an alert() in the web page you are debugging if you
    attempt to debug it with weinre and you are using an unsupported
    version of Prototype.js.

Platforms supported - debug server

Any platform that supports node.js.

Platforms supported - debug client

The browser where the debugger user interface runs.

  • Google Chrome
  • Apple Safari
  • Other recent-ish WebKit-based browsers

Platforms supported - debug target

The browser with the page you are debugging.

  • Android Browser application
  • iOS Mobile Safari application
  • PhoneGap/Cordova
  • other

Platforms not supported - debug target

  • iOS 3.1.3 or earlier
  • webOS 1.45 or earlier

2012年5月6日星期日

Who They Are

Who We Are - DocumentCloud

DocumentCloud is a catalog of primary source documents and a tool for annotating, organizing and publishing them on the web. Documents are contributed by journalists, researchers and archivists. We're helping reporters get more out of documents and helping newsrooms make their online presence more engaging.

DocumentCloud was founded in 2009 with a grant from the Knight News Challenge. After two years as an independent nonprofit organization, DocumentCloud became a project of Investigative Reporters and Editors in June of 2011.

DocumentCloud Staff

IRE's staff are all actively involved in helping guide DocumentCloud's next stage of development.

Ted Han
Ted Han joins DocumentCloud from Videojuicer, an online video platform focused on open standards and software integration. He's a computational linguist by degree, developer by trade, and Sci-Fi nerd by leisure. He was selected as a participant in the Knight Mozilla Journalism Challenge and has worked on DataMapper, Merb and a variety of data-based projects, including the CrisisCommons response to the Tohoku earthquake & tsunami.

Lauren Grandestaff, director of IRE's Resource Center, provides support to DocumentCloud users. She directs collection, indexing, and archiving of research materials that include tens of thousands of investigative stories and thousands of reporter tipsheets and coordinates IRE's annual awards, which honor the best investigative journalism around the world. A lifelong Missourian, Lauren does her part to keep Columbia, Missouri's local music scene thriving, too.

Advisory Group

DocumentCloud's founders and founding staff continue to advise us on the project.

Jeremy Ashkenas
Jeremy Ashkenas, Lead Developer 2009-2011, joined DocumentCloud from
Zenbe, a provider of online email and
collaboration software. He created the
Ruby-Processing
visualization toolkit, the
CoffeeScript
programming language and is a two-time winner of the Sunlight Foundation’s
Apps for America competition, for
Know Thy Congressman and
Quakespotter. You can contact him
at jeremy@documentcloud.org.

 

Amanda Hickman
Amanda Hickman, Program Director 2009-2011, joined DocumentCloud from
Gotham Gazette where, as the
Director of Technology, she managed development of a series of
games about public
policy
issues, built a pretty cool
database of candidates for
local office
and shared an
ONA award for General Excellence
with her colleagues there. Prior to joining Gotham Gazette, she worked as a
Circuit Rider,
providing technology assistance and training to low-income grassroots groups
in the U.S. and as a consultant to foundations
looking for ways to support their grantees’ use of technology in organizing
work. She taught an undergraduate course at NYU’s Gallatin School on using
the Internet as an organizing tool. You can find her at
amanda@documentcloud.org.

Scott Klein
is the Editor of News Applications at ProPublica,
directing news application development and production. He previously worked
at The Nation, directing editorial and business application
development for the TheNation.com.
Earlier in his career, Scott worked at The New York Times.

Aron Pilhofer is editor of Interactive News at The New York Times. His
team blends journalism and technology to enhance The Times's reporting
online through social media, community and dynamic, data-driven Web
applications. He is also co-founder of two news-related nonprofits:
DocumentCloud.org and Hacks and Hackers. Aron joined The Times in 2005
as a projects editor on the paper's newly expanded computer-assisted
reporting team. Prior to joining The Times, he reported for the Center
for Public Integrity in Washington, Gannett newspapers in New Jersey and
Delaware and was on the national training staff of Investigative
Reporters and Editors
.

Eric Umansky
is a senior editor at ProPublica,
where he oversees daily reporting. Previously, Eric wrote Slate’s
“Today’s Papers”
He also written, mostly on national security issues for
The New York Times Magazine, Washington Post,
The New Republic, and elsewhere. Earlier in his career, Eric was
editor of MotherJones.com.

a colorful termial output for phantomjs porting from casperjs


var fs = require('fs');

    var options    = { bold: 1, underscore: 4, blink: 5, reverse: 7, conceal: 8 };
    var foreground = { black: 30, red: 31, green: 32, yellow: 33, blue: 34, magenta: 35, cyan: 36, white: 37 };
    var background = { black: 40, red: 41, green: 42, yellow: 43, blue: 44, magenta: 45, cyan: 46, white: 47 };
    var styles     = {
        'ERROR':     { bg: 'red', fg: 'white', bold: true },
        'INFO':      { fg: 'green', bold: true },
        'TRACE':     { fg: 'green', bold: true },
        'PARAMETER': { fg: 'cyan' },
        'COMMENT':   { fg: 'yellow' },
        'WARN':   { fg: 'red', bold: true },
        'GREEN_BAR': { fg: 'white', bg: 'green', bold: true },
        'RED_BAR':   { fg: 'white', bg: 'red', bold: true },
        'INFO_BAR':  { bg: 'cyan', fg: 'white', bold: true }
    };

    /**
     * Adds a style to provided text.
     *
     * @param   String  text
     * @param   String  styleName
     * @return  String
     */
    function colorize(text, styleName, pad) {

        return format(text, styles[styleName], pad);
    };

    /**
     * Formats a text using a style declaration object.
     *
     * @param  String  text
     * @param  Object  style
     * @return String
     */
    function format(text, style, pad) {
        //if (fs.isWindows() || !isObject(style)) {
        //    return text;
        //}
        var codes = [];
        if (style.fg && foreground[style.fg]) {
            codes.push(foreground[style.fg]);
        }
        if (style.bg && background[style.bg]) {
            codes.push(background[style.bg]);
        }
        for (var option in options) {
            if (style[option] === true) {
                codes.push(options[option]);
            }
        }
        // pad
        if (typeof pad === "number" && text.length < pad) {
            text += new Array(pad - text.length + 1).join(' ');
        }
        return "\033[" + codes.join(';') + 'm' + text + "\033[0m";
    };






var pad=0;
console.log(colorize("aaa", "ERROR"));
console.log("zzz");
phantom.exit(0);

They're hiring based on New Brunswick, NJ, USA

This is a professional company focus on data extract & collection,but i think they technoloy backgroup is not strong enough,

more importanly is a sales-marketing driven company

i got this infomation from my ad-words gmail in-box

Connotate Software Engineer | SmartRecruiters Free recruiting software for applicant management

Software Engineer

New Brunswick, NJ, USA

REF9T

 


 

Company Description

Connotate is a venture backed company that has revolutionized the collection and monitoring of web data with our patented software algorithms.  At Connotate, we share common goals and work together to achieve success.  We are passionate about what we do.  If you’re ready to make a difference at a high-growth emerging tech company, look no further.

 

Job Description

**Local Candidates Only**

 

The Connotate Engineering team is looking for people who are smart, energetic, open and agile, who embrace a dynamic environment, love to solve difficult technical problems, have a passion for the team dynamic, lead by example, and who are excited by challenges in the area of web and massive data sets.

 

Overall Responsibility

Develop Connotate’s next generation products focusing on all aspects of data monitoring and mining, including the GUI-based studio in which data mining agents are designed, data monitoring and extraction methods, and the management of agents as they run on premise or in the Cloud. 

 

Key Areas of Responsibility

  • Produce high-quality code within the Agile software development process (Scrum).
  • Develop new products, features and enhancements, including test driven development, unit testing, and documentation.
  • Collaborate with software engineers, testers, product owners, and roles in other areas of the company.
  • Work with C#, ASP.NET, WPF, WCF, XML/XSLT/XPath, Web Services, nUnit and SQL Server (including developing SQL queries, stored procedures, functions and triggers).
  • Work with SVN, Cruise Control, and Install Shield for source control and build activities.

 

Qualifications

The candidate has experience with many of the following items:

  • C#, ASP.Net, WPF, WCF, XML/XSLT/XPath, Web Services SQL Server (including developing SQL queries, stored procedures, functions and triggers)
  • Multi process and multi threading programming
  • SVN for Source control
  • 3 years of related software development experience
  • Software as a Service (SaaS) and Cloud Computing is a plus
  • Machine learning is a plus
  • Big Data is a plus
  • Agile (Scrum) is a plus
  • BS Computer Science or related field, Masters Degree is a plus

 

Additional Information

The candidate must be able to demonstrate the following personal skills as they apply to the performance of duties:

  • Good communication skills
  • Ability to work in a dynamic, collaborative environment
  • Capability of handling several deadline-driven tasks at once
  • Possesses a high level of comfort dealing with sometimes shifting priorities
  • Ability to see the ultimate goal (big picture) while focusing on the details

 

2012年5月5日星期六

2012年5月4日星期五

HOWTO: Secure Firefox and IM with PuTTY | Thinkhole Labs

HOWTO: Secure Firefox and IM with PuTTY | Thinkhole Labs

HOWTO: Secure Firefox and IM with PuTTY

There are times when you want to connect to the Internet through unknown and/or insecure networks such as the local Panera or other WiFi hotspot. If you aren’t careful, you might make it all too easy for someone to sniff your connection using Ettercap.

One of the best ways to secure your connection is to use a VPN, but that isn’t always practical. So here’s a way to securely connect to the net using only an SSH client and a remote box that you control/trust.

Requirements:

  1. PuTTY* loaded on your local machine
  2. Remote host running OpenSSH (e.g. Linux box at home)
  3. Firefox (obviously)
  4. Gaim for all your IM needs

Just follow these steps…



1. Create a new PuTTY session

Run PuTTY and create a new session in PuTTY to connect to the remote host that is running OpenSSH. Fill in the hostname, the port (usually 22), make sure SSH is checked, give it a session name and hit Save:

PuTTY Session Config

2. Configure a secure tunnel

Click on “Tunnels” on the left and set up dynamic fowarding for a local port (e.g. 7070). Under “Add new forwarded port” type in 7070 for the source port, leave the destination blank, and check Auto and Dynamic. Then it the Add button. If you did it correctly, you’ll see D7070 listed in the Forwarded Ports box:

PuTTY Tunnels Config

That’s it for tunnels, as there is no need to create more than one. Remember to save your session profile in PuTTY so you don’t have to set up the tunnel next time.

3. Connect to the remote SSH box

Double click on the connection profile and type in your username and password when prompted.

4. Configure Firefox

Go to Tools, Options, General, and then click on Connection Settings…

Firefox connections

Check Manual Proxy Configuration, leave most of the fields blank, but fill in 127.0.0.1 for the SOCKS v5 host with a port of 7070 (or whatever you used in Step 2):

Firefox connection settings

5. Configure Gaim

Fire up Gaim and hit the Preferences button:

Gaim

Then select Network on the left and set up the Proxy Server. The Proxy Type should be SOCKS 5. The host is 127.0.0.1 and the port is 7070 (or whatever you chose in Step 2).

Gaim

There’s no need for a user or password. Then hit close.

6. Enjoy

That’s it. From now on, as long as you first log into the remote ssh host with PuTTY, your Firefox and IM traffic will be routed over a secure tunnel to the remote host and then out to the Net. Good stuff.

* Yes, PuTTY is available for Linux. It’s even in Portage!

Update (Email): Actually, the setup for Thunderbird to securely proxy your email traffic is pretty much the same as it is for Firefox.

And then type in 127.0.0.1 and your port number:

That’s it.

Update: Getting some linkage from Digg.

Corrections/Addendum:

  1. Note that this method will secure your connection between your remote location (e.g. WiFi hotspot) and the ssh host (e.g. Linux box at home). It is not secure from the ssh host to Internet. For the most part, that’s OK as it will provide reasonable protection from people running packet sniffers at the hotspot. But please recognize that if your ssh host is on your cable connection at home, your ISP can still (obviously) easily sniff all your packets.
  2. Quite a few people have correctly pointed out that DNS queries will still be “leaked” to the untrusted network. So the names of any sites you visit will still get logged.

    Now if you don’t mind people knowing what sites your are connecting to, then there’s nothing to worry about. But if are running the current version of Firefox and would like to protect that information, you can open the about:config page, and change network.proxy.socks_remote_dns to true.

    You can do the same thing in Thunderbird if you would like.

    For a greater level of security on all your connections, you should consider running a full VPN (see also: IPCop + OpenVPN HOWTO).

  3. As Nate pointed out in the comments, if you have the command-line version of OpenSSH already installed on your computer, you don’t need to mess with PuTTY. Just run ssh -D 7070 user@host.example.com and that will set up your tunnel. I’m not an Apple user, but I think OS X has everything you need.
  4. There are portable versions of Firefox, Gaim, and Thunderbird, and PuTTY runs from a USB drive. So using this method (unlike using OpenVPN) there is no need to install anything on the computer you are using. Quite handy if you are borrowing someone’s computer or you are in a school computer lab.
  5. How do you know if it’s working? Personally, I used SmartSniff to look at the packets and make sure they looked encrypted and were on the right ports. Of course, any packet sniffer would do.

    The other method (and I tried this one too) is to get all set up and running with the tunnels. And then after it is apparently working, kill PuTTY and see if you lose the connection.

    Of the two, the packet sniffer is the better way to go.

  6. Apparently Opera only does SOCK4. I didn’t personally try it, but this should work with SOCKS4 proxies as well as SOCKS5.
  7. SocksCap (non-commercial, home-use only) will let you run just about anything over a SOCKS5 proxy.

2012年5月3日星期四

crawler 0.1.0

crawler 0.1.0 : Python Package Index

crawler 0.1.0

python crawler.

Latest Version:
0.1.2

python crawler.
=====
## Example
=====

from crawler.crawler import Crawler

mycrawler = Crawler()
seeds = ['http://www.example.com/'] # list of url
mycrawler.add_seeds(seeds)
url_patterns = ['^(.+example\.com)(.+)

] # list of regular expression for urls that crawler will work on. mycrawler.start(url_patterns) # start crawling ################# data files ################# three database (Berkeley DB) files will be generated. queue.db webpage.db duplcheck.db