ieHtml - identifying information

I am starting to use the above control to show some web site within my application. They provide specific searches and one of the sites would like to have a way to identify that it is my program which is accessing their site to provide some special processing.

They are proposing for me to set the 'HTTP_USER_AGENT' header.

Would it be possible to append some identifying string to that when I call the site?

I use:
self.ie = iewin.IEHtmlWindow(self)
....
self.ie.Navigate(url)

Doing some googling and looking at the lib.iewin.py code I see that Navigate has a "Headers" parameter, but I have no clue how that would be used and if this could help in setting some variable the web site could use for the special processing.

Anyone can give me some hints/tips?

Thanks in advance
Werner

Werner F. Bruhin wrote:

I am starting to use the above control to show some web site within my application. They provide specific searches and one of the sites would like to have a way to identify that it is my program which is accessing their site to provide some special processing.

They are proposing for me to set the 'HTTP_USER_AGENT' header.

Would it be possible to append some identifying string to that when I call the site?

I use:
self.ie = iewin.IEHtmlWindow(self)
....
self.ie.Navigate(url)

Doing some googling and looking at the lib.iewin.py code I see that Navigate has a "Headers" parameter, but I have no clue how that would be used and if this could help in setting some variable the web site could use for the special processing.

Anyone can give me some hints/tips?

Thanks in advance
Werner

I'm not finding much either. You may be able to fake it using urllib2: Python Requests Module: Sending Requests to APIs and Handling Responses

Otherwise, I would check MSDN and or Google around for how they do it when they embed IE in a VB app.

···

-------------------
Mike Driscoll

Blog: http://blog.pythonlibrary.org
Python Extension Building Network: http://www.pythonlibrary.org

Mike,

Mike Driscoll wrote:

Werner F. Bruhin wrote:

I am starting to use the above control to show some web site within my application. They provide specific searches and one of the sites would like to have a way to identify that it is my program which is accessing their site to provide some special processing.

They are proposing for me to set the 'HTTP_USER_AGENT' header.

Would it be possible to append some identifying string to that when I call the site?

I use:
self.ie = iewin.IEHtmlWindow(self)
....
self.ie.Navigate(url)

Doing some googling and looking at the lib.iewin.py code I see that Navigate has a "Headers" parameter, but I have no clue how that would be used and if this could help in setting some variable the web site could use for the special processing.

Anyone can give me some hints/tips?

Thanks in advance
Werner

I'm not finding much either. You may be able to fake it using urllib2: Python Requests Module: Sending Requests to APIs and Handling Responses

The link is interesting info, but I do not think this works with iewin.

Otherwise, I would check MSDN and or Google around for how they do it when they embed IE in a VB app.

I just went to the trial and error approach and was lucky, it is very easy actually (isn't it always with wxPython :slight_smile: ).

Just changed my Navigate call to this:
        xHeader = 'Referer: http://www.thewinecellarbook.com'
        self.searchResult.Navigate(url, Headers=xHeader)

I guess above could also be used for the user agent.

Maybe this could go into the wxPython demo or into the documentation of iewin?

Werner

···

-------------------
Mike Driscoll

Blog: http://blog.pythonlibrary.org
Python Extension Building Network: http://www.pythonlibrary.org

_______________________________________________
wxpython-users mailing list
wxpython-users@lists.wxwidgets.org
http://lists.wxwidgets.org/mailman/listinfo/wxpython-users

Werner F. Bruhin wrote:

Mike,

Mike Driscoll wrote:

Werner F. Bruhin wrote:

I am starting to use the above control to show some web site within my application. They provide specific searches and one of the sites would like to have a way to identify that it is my program which is accessing their site to provide some special processing.

They are proposing for me to set the 'HTTP_USER_AGENT' header.

Would it be possible to append some identifying string to that when I call the site?

I use:
self.ie = iewin.IEHtmlWindow(self)
....
self.ie.Navigate(url)

Doing some googling and looking at the lib.iewin.py code I see that Navigate has a "Headers" parameter, but I have no clue how that would be used and if this could help in setting some variable the web site could use for the special processing.

Anyone can give me some hints/tips?

Thanks in advance
Werner

I'm not finding much either. You may be able to fake it using urllib2: Python Requests Module: Sending Requests to APIs and Handling Responses

The link is interesting info, but I do not think this works with iewin.

Otherwise, I would check MSDN and or Google around for how they do it when they embed IE in a VB app.

I just went to the trial and error approach and was lucky, it is very easy actually (isn't it always with wxPython :slight_smile: ).

Just changed my Navigate call to this:
       xHeader = 'Referer: http://www.thewinecellarbook.com'
       self.searchResult.Navigate(url, Headers=xHeader)

I guess above could also be used for the user agent.

Maybe this could go into the wxPython demo or into the documentation of iewin?

Werner

I think you should just stick it in the wiki. Then it's in at least one location. If Robin also adds it to the docs, that's great too.

Mike

Mike,

Mike Driscoll wrote:
...

I just went to the trial and error approach and was lucky, it is very easy actually (isn't it always with wxPython :slight_smile: ).

Just changed my Navigate call to this:
       xHeader = 'Referer: http://www.thewinecellarbook.com'
       self.searchResult.Navigate(url, Headers=xHeader)

I guess above could also be used for the user agent.

Maybe this could go into the wxPython demo or into the documentation of iewin?

Werner

I think you should just stick it in the wiki. Then it's in at least one location. If Robin also adds it to the docs, that's great too.

Good idea.
Done: wx.lib.iewin - wxPyWiki

I hope it is linked in about in the right place under RecipesControls.

Werner

Werner F. Bruhin wrote:

Mike,

Mike Driscoll wrote:
...

I just went to the trial and error approach and was lucky, it is very easy actually (isn't it always with wxPython :slight_smile: ).

Just changed my Navigate call to this:
       xHeader = 'Referer: http://www.thewinecellarbook.com'
       self.searchResult.Navigate(url, Headers=xHeader)

I guess above could also be used for the user agent.

Maybe this could go into the wxPython demo or into the documentation of iewin?

Werner

I think you should just stick it in the wiki. Then it's in at least one location. If Robin also adds it to the docs, that's great too.

Good idea.
Done: wx.lib.iewin - wxPyWiki

I hope it is linked in about in the right place under RecipesControls.

Werner

Looks ok to me. Someone really should do some more organizing on the wiki though. It's kind of hard to find what one is looking for. One example would be to have sub-categories in the Control page where each widget would have a header and the recipes would be nested under the header, like this:

- Validators:
   - Creating Validators Based on State Machines, Part I
   - Creating Validators Based on State Machines, Part II

I assume Moinmoin allows nested lists...

Anyway, thanks for the info. Hopefully it will be useful to the rest of us on Windows!

Mike

Werner F. Bruhin wrote:

Mike,

Mike Driscoll wrote:
...

I just went to the trial and error approach and was lucky, it is very easy actually (isn't it always with wxPython :slight_smile: ).

Just changed my Navigate call to this:
       xHeader = 'Referer: http://www.thewinecellarbook.com'
       self.searchResult.Navigate(url, Headers=xHeader)

The param is called "Headers" so I assume one can set one or more header. Is this correct and if yes what is the syntax to set multiple header values.

And one more question, is it possible to append information to an existing header. If yes, what would be the syntax for that?

Werner

Werner F. Bruhin wrote:

Werner F. Bruhin wrote:

Mike,

Mike Driscoll wrote:
...

I just went to the trial and error approach and was lucky, it is very easy actually (isn't it always with wxPython :slight_smile: ).

Just changed my Navigate call to this:
       xHeader = 'Referer: http://www.thewinecellarbook.com'
       self.searchResult.Navigate(url, Headers=xHeader)

The param is called "Headers" so I assume one can set one or more header. Is this correct and if yes what is the syntax to set multiple header values.

Still not found an answer, I passed multiple headers in as a list, a string, a dict (which throws and error) and as a urllib2.encoded structure and none of them work, i.e. only one header can be set.

I would think if multiple are not possible that the parameter would be called "Header" and not "Headers", so there must be some other format/syntax which should work.

BTW, one also seems only to be able to set "HTTP_something" headers, i.e. any name used not defined as such is just quietly being ignored.

I would really appreciate if anyone can shed some light on this puzzle.

Werner

Werner F. Bruhin wrote:

Werner F. Bruhin wrote:

Werner F. Bruhin wrote:

Mike,

Mike Driscoll wrote:
...

I just went to the trial and error approach and was lucky, it is very easy actually (isn't it always with wxPython :slight_smile: ).

Just changed my Navigate call to this:
       xHeader = 'Referer: http://www.thewinecellarbook.com'
       self.searchResult.Navigate(url, Headers=xHeader)

The param is called "Headers" so I assume one can set one or more header. Is this correct and if yes what is the syntax to set multiple header values.

Still not found an answer, I passed multiple headers in as a list, a string, a dict (which throws and error) and as a urllib2.encoded structure and none of them work, i.e. only one header can be set.

I would think if multiple are not possible that the parameter would be called "Header" and not "Headers", so there must be some other format/syntax which should work.

BTW, one also seems only to be able to set "HTTP_something" headers, i.e. any name used not defined as such is just quietly being ignored.

I would really appreciate if anyone can shed some light on this puzzle.

Forgot to include the error I get when I use a dict:
  File "C:\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\lib\iewin.py", line 166, in Navigate
    return self.ctrl.Navigate2( URL, Flags, TargetFrameName, PostData, Headers)
ArgumentError: argument 5: <type 'exceptions.TypeError'>: Cannot put {'PNV_PARTNER': 'TWCBEMBEDDED', 'from': 'wbruhin@thewinecellarbook.com'} in VARIANT

Werner

Hi Werner,

You may already know this, but the iehtml control is basically exposing
the IWebBrowser2 interface that is documented here:

Having said that, the documentation for the Navigate method doesn't
actually explain how to pass multiple headers :frowning:

Have you tried passing a string in which the custom headers are
separated by a carriage return and a line feed (ie. '\r\n')?

Also note that if the page references other resources (such as images,
stylesheets, javascript etc), those requests won't use your custom user
agent. There do seem to be some suggestions on the web that by handling
the BeforeNavigate event, you can insert your header at that point.

Hope that helps,

Simon

···

-----Original Message-----
From: wxpython-users-bounces@lists.wxwidgets.org
[mailto:wxpython-users-bounces@lists.wxwidgets.org] On Behalf
Of Werner F. Bruhin
Sent: 23 January 2009 10:12
To: wxpython-users@lists.wxwidgets.org
Subject: Re: [wxpython-users] ieHtml - identifying information

Werner F. Bruhin wrote:
> Werner F. Bruhin wrote:
>> Werner F. Bruhin wrote:
>>> Mike,
>>>
>>> Mike Driscoll wrote:
>>> ...
>>>>> I just went to the trial and error approach and was
lucky, it is
>>>>> very easy actually (isn't it always with wxPython :slight_smile: ).
>>>>>
>>>>> Just changed my Navigate call to this:
>>>>> xHeader = 'Referer: http://www.thewinecellarbook.com'
>>>>> self.searchResult.Navigate(url, Headers=xHeader)
>> The param is called "Headers" so I assume one can set one or more
>> header. Is this correct and if yes what is the syntax to set
>> multiple header values.
> Still not found an answer, I passed multiple headers in as
a list, a
> string, a dict (which throws and error) and as a urllib2.encoded
> structure and none of them work, i.e. only one header can be set.
>
> I would think if multiple are not possible that the
parameter would be
> called "Header" and not "Headers", so there must be some other
> format/syntax which should work.
>
> BTW, one also seems only to be able to set "HTTP_something"
headers,
> i.e. any name used not defined as such is just quietly
being ignored.
>
> I would really appreciate if anyone can shed some light on
this puzzle.
Forgot to include the error I get when I use a dict:
  File
"C:\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\lib\iewin
.py", line
166, in Navigate
    return self.ctrl.Navigate2( URL, Flags, TargetFrameName,
PostData,
Headers)
ArgumentError: argument 5: <type 'exceptions.TypeError'>: Cannot put
{'PNV_PARTNER': 'TWCBEMBEDDED', 'from':
'wbruhin@thewinecellarbook.com'}
in VARIANT

Werner

Simon,

King Simon-NFHD78 wrote:

From: wxpython-users-bounces@lists.wxwidgets.org [mailto:wxpython-users-bounces@lists.wxwidgets.org] On Behalf Of Werner F. Bruhin
Sent: 23 January 2009 10:12
To: wxpython-users@lists.wxwidgets.org
Subject: Re: [wxpython-users] ieHtml - identifying information

Werner F. Bruhin wrote:
    

Werner F. Bruhin wrote:
      

Werner F. Bruhin wrote:
        

Mike,

Mike Driscoll wrote:
...
          

I just went to the trial and error approach and was
              

lucky, it is
    

very easy actually (isn't it always with wxPython :slight_smile: ).

Just changed my Navigate call to this:
       xHeader = 'Referer: http://www.thewinecellarbook.com'
       self.searchResult.Navigate(url, Headers=xHeader)
              

The param is called "Headers" so I assume one can set one or more header. Is this correct and if yes what is the syntax to set multiple header values.
        

Still not found an answer, I passed multiple headers in as
      

a list, a
    

string, a dict (which throws and error) and as a urllib2.encoded structure and none of them work, i.e. only one header can be set.

I would think if multiple are not possible that the
      

parameter would be
    

called "Header" and not "Headers", so there must be some other format/syntax which should work.

BTW, one also seems only to be able to set "HTTP_something"
      

headers,
    

i.e. any name used not defined as such is just quietly
      

being ignored.
    

I would really appreciate if anyone can shed some light on
      

this puzzle.
Forgot to include the error I get when I use a dict:
  File "C:\Python25\lib\site-packages\wx-2.8-msw-unicode\wx\lib\iewin
.py", line 166, in Navigate
    return self.ctrl.Navigate2( URL, Flags, TargetFrameName, PostData, Headers)
ArgumentError: argument 5: <type 'exceptions.TypeError'>: Cannot put {'PNV_PARTNER': 'TWCBEMBEDDED', 'from': 'wbruhin@thewinecellarbook.com'} in VARIANT

Werner
    
Hi Werner,

You may already know this, but the iehtml control is basically exposing
the IWebBrowser2 interface that is documented here:

IWebBrowser2 interface (Windows)
  

Yes, I have looked through this.

Having said that, the documentation for the Navigate method doesn't
actually explain how to pass multiple headers :frowning:

Have you tried passing a string in which the custom headers are
separated by a carriage return and a line feed (ie. '\r\n')?
  

Great! Yes, this allows me to pass multiple headers (e.g. HTTP_REFERER and HTTP_FROM work nicely).

Now I just need to figure out how I can append an identifier to the user_agent. The "HTTP_USER_AGENT" can not be set through the "Headers" parameter (at least I can't make it work).

The site I am working with (not in my control) needs a header which is persistent across page changes by the user and so far that is the only one we can see doing this.

Also note that if the page references other resources (such as images,
stylesheets, javascript etc), those requests won't use your custom user
agent. There do seem to be some suggestions on the web that by handling
the BeforeNavigate event, you can insert your header at that point.
  

That does not seem to work, but maybe I am doing something wrong here.

Hope that helps,
  

Yes, it does - also I am still not quite there.

Thanks
Werner

···

-----Original Message-----

I haven't used the control myself, so this may not work, but this page
is one I found while looking for the answer to your problem:

http://wallerdev.com/2008/12/25/changing-the-user-agent-in-ie-net-webbro
wser-control-via-csharp

Firstly, it implies that setting the user agent via a custom header
really should work. Are you using the format "User-Agent:
<your-user-agent>\r\n"?

Secondly, it overrides the BeforeNavigate method to insert the extra
header. Going by the ActiveX_IEHtmlWindow wxPython demo, you should be
able to do this by creating a method called BeforeNavigate, and calling
'self.ctrl.AddEventSink(self)'. I think you can ignore the other
methods.

(The demo actually implements BeforeNavigate2 - I don't know whether you
need to do that, or whether BeforeNavigate is enough)

Once you've ensured your BeforeNavigate method is being called, you
should be able to cancel the navigation and start another one that
includes your custom User-Agent header (be aware that it will result in
your BeforeNavigate method being called again - you need to store some
kind of state to ensure that you don't cancel your own navigation
attempt! That's what the 'renavigating' variable is for in the web page
above)

I imagine that this still won't allow for custom headers in requests for
images etc., but perhaps you can get away with that.

I'm still looking forward to wxWebkit - having a cross-platform browser
solution for wxPython will be fantastic, and I'm sure it'll be much more
flexible than embedding IE as well.

Hope that helps,

Simon

···

-----Original Message-----
From: wxpython-users-bounces@lists.wxwidgets.org
[mailto:wxpython-users-bounces@lists.wxwidgets.org] On Behalf
Of Werner F. Bruhin
Sent: 23 January 2009 11:38
To: wxpython-users@lists.wxwidgets.org
Subject: Re: [wxpython-users] ieHtml - identifying information

Simon,
>
> Hi Werner,
>
> You may already know this, but the iehtml control is
basically exposing
> the IWebBrowser2 interface that is documented here:
>
> IWebBrowser2 interface (Windows)
>
Yes, I have looked through this.
> Having said that, the documentation for the Navigate method doesn't
> actually explain how to pass multiple headers :frowning:
>
> Have you tried passing a string in which the custom headers are
> separated by a carriage return and a line feed (ie. '\r\n')?
>
Great! Yes, this allows me to pass multiple headers (e.g.
HTTP_REFERER
and HTTP_FROM work nicely).

Now I just need to figure out how I can append an identifier to the
user_agent. The "HTTP_USER_AGENT" can not be set through the
"Headers"
parameter (at least I can't make it work).

The site I am working with (not in my control) needs a header
which is
persistent across page changes by the user and so far that is
the only
one we can see doing this.
> Also note that if the page references other resources (such
as images,
> stylesheets, javascript etc), those requests won't use your
custom user
> agent. There do seem to be some suggestions on the web that
by handling
> the BeforeNavigate event, you can insert your header at that point.
>
That does not seem to work, but maybe I am doing something wrong here.
> Hope that helps,
>
Yes, it does - also I am still not quite there.

Thanks
Werner

Simon,
King Simon-NFHD78 wrote:
...

I haven't used the control myself, so this may not work, but this page
is one I found while looking for the answer to your problem:

http://wallerdev.com/2008/12/25/changing-the-user-agent-in-ie-net-webbro
wser-control-via-csharp

Firstly, it implies that setting the user agent via a custom header
really should work. Are you using the format "User-Agent:
<your-user-agent>\r\n"?
  

OK the format was the problem, i.e. I used "user_agent: ..." but it has to be "user-agent".

Simon, thanks a lot!

Will update the wiki with these findings so that others will have an easier time.

...

I'm still looking forward to wxWebkit - having a cross-platform browser
solution for wxPython will be fantastic, and I'm sure it'll be much more
flexible than embedding IE as well.
  

Yes, I am looking forward to the wxWebkit too, just using the IE control for the meantime.

Hope that helps,
  

It did a lot!

Thanks again
Werner