3

BACKGROUND: The script below logs into my salesforce.com account and tries to click a bunch of "download" links and save the resulting .zip export files locally.

I'm using jruby and celerity (which is a jruby wrapper around htmlunit, a headless browser, which also supports javascript).

The requirement is that it runs from a LINUX command line without a gui/browser installed. The headless browser also needs to support javascript otherwise salesforce.com returns an error and it will not render the download links. (I tried CURL but it fails since it does not support javascript). (Currently the script works fine when run from the command line on a WINDOWS box)

QUESTION: When i try to run this from a linux box, i receive the error below and the script does not appear to login to salesforce. i believe the .js warning is ok (i think), but i don't understand what the VBScript not supported in Window.execScript() warning means?:

[root@ip-10-114-241-55 ~]# jruby download_sf_export_files.rb
Jan 18, 2011 2:16:41 AM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Expected content type of 'application/javascript' or 'application/ecmascript' for remotely loaded JavaScript element at 'https://www.salesforce.com/common/assets/js/global2.js?date=100817', but got 'application/x-javascript'.
Jan 18, 2011 2:16:41 AM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Expected content type of 'application/javascript' or 'application/ecmascript' for remotely loaded JavaScript element at 'https://www.salesforce.com/common/assets/thirdparty/omniture/s_code3.js?date=101012', but got 'application/x-javascript'.
Jan 18, 2011 2:38:05 AM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Expected content type of 'application/javascript' or 'application/ecmascript' for remotely loaded JavaScript element at 'https://na7.salesforce.com/static/101210/js/functions.js', but got 'application/x-javascript'.
Jan 18, 2011 2:38:06 AM com.gargoylesoftware.htmlunit.javascript.host.Window jsxFunction_execScript
WARNING: VBScript not supported in Window.execScript().



-----install instructions------
download jruby:                 http://jruby.org/download
install gem from command line:  jruby -S gem install celerity
run script from command line:   jruby download_sf_export_files.rb

-----download_sf_export_files.rb-------
require "rubygems"
require "celerity"

browser = Celerity::Browser.new
browser.goto('https://na7.salesforce.com')
browser.text_field(:name, 'username').value = 'myuserlogin@domain.com'
browser.text_field(:name, 'pw').value = 'mysecretpassword'
browser.button(:name, 'Login').click
browser.goto('https://na7.salesforce.com/ui/setup/export/DataExportPage/d')

## assume multiple 'download' links if a large SF data export 
## group all links in a map, iterate and click each one

i=1
hrefs = browser.links.map { |link| link.href if link.text =~ /download/ }.compact hrefs.each do |href|
 browser.link(:url, href).download
 puts "found a 'download' link on page, clicking link..."
 io = browser.link(:text, "download").download 
 File.open("sf_zip_file_#{i}.zip", "w+") { |file| file.write(io.read) }
 i+=i
end
henrycarteruk
  • 12,708
  • 2
  • 36
  • 40
solidlight
  • 61
  • 5
  • Sorry don't have a SF account in which to duplicate the problem, but will give you a tip: `Enumerable#each_with_index` allows you to get rid of the separate counter – Mark Thomas Jan 20 '11 at 01:16
  • Can you try to capture the page Celerity "sees" after your login attempt? Maybe there's some error mentioned, maybe you could examine HTTP header you got back, verify if you have landed on address of Home page... I smell some security settings (machine with black-listed IP or perhaps this screen when you need to receive verification email and click link that comes in with Celerity to set cookie). – eyescream Jan 20 '11 at 06:07
  • Thanks Mark for the tip. looks like .each_with_index is supported in ruby 1.9 and in 1.8.6 you use .to_enum(:each_with_index) – solidlight Jan 20 '11 at 10:30
  • eyescream, you guessed it! problem was the SF ip activation page. however, i still get the .js and .vb warnings... i saved browser.html to a file to to inspect the code as you suggested. then added the following to click the activation button for me: `if browser.text.include? "You are attempting to access salesforce.com from an unrecognized computer." browser.button(:value, 'Send Activation Link').click` (what mislead me was the fact that i could access the SF API with now problem, which is of course a different access path than the web login.) thank you! – solidlight Jan 20 '11 at 10:42
  • You're welcome :) In API the security token is equivalent of this cookie setting page... As for the actual JS/VB errors - probably you don't have to worry about them if functions work anyway? Maybe try to set Celerity to work in somewhat less strict mode? No idea :) – eyescream Jan 20 '11 at 14:29
  • eyescream - could you move your comment down to be an answer? You'll get the points that way, and this won't show up as an unanswered question :-) Thanks! – metadaddy Jan 20 '11 at 17:52
  • darn although now it looks like the `.download` method is not supported on linux, it's part of celerity' link method: `http://celerity.rubyforge.org/yard/Celerity/ClickableElement.html#download-instance_method`, but i guess that's for another question :) – solidlight Jan 20 '11 at 23:24
  • SOLVED: i had the wrong version of the celerity gem on the linux box, because it had the wrong gem sources url, doh! some basics: to add a source: `jruby -S gem sources -a http://gems.rubyforge.org/` to check which source is being used: `jruby -S gem env` and to list installed gems: `jruby -S gem list`. once i got the correct gem `celerity (0.8.7)`, the `.dowload` method worked on linux. i'm not sure what the .js or .vb warnings are, but i can turn the log_level off in celerity with either `browser = Celerity::Browser.new(:log_level => :off)` or `browser.log_level = :off`. – solidlight Jan 21 '11 at 21:53
  • Is the script in the question updated with your findings? Would like to snatch it and try :) – Buzzzz Mar 11 '11 at 12:41
  • Thanks a lot my friend! Thanks to this post I didn't five up and made successful attempt of connecting to SF with Java + HtmlUnit – ŁukaszBachman Jun 01 '12 at 13:30

1 Answers1

3

(Copied from question comments just to remove it from "unanswered questions")

Can you try to capture the page Celerity "sees" after your login attempt? Maybe there's some error mentioned, maybe you could examine HTTP header you got back, verify if you have landed on address of Home page... I smell some security settings (machine with black-listed IP or perhaps this screen when you need to receive verification email and click link that comes in with Celerity to set cookie).

eyescream
  • 18,088
  • 2
  • 34
  • 46