issue59: Proxy management API

Priority: wish Status: chatting
Messages
msg77 (view) Author: nicktastic Date: 2008-10-02.17:34:25
*** Proxy API RFC


** API

define_hook("proxy_enabled_hook");
define_hook("proxy_disabled_hook");
define_hook("proxy_profile_updated_hook");

define_keywords("$http", "$ssl", "$ftp", "$gopher", "$socks", "$socks_ver",
                "$socks_dns", "$excludes" "$ac_url", "$ac_retry_min",
                "$ac_retry_max", "$ac_timeout");
function proxy_define_profile(name) {}

define_keywords("$http", "$ssl", "$ftp", "$gopher", "$socks", "$socks_ver",
                "$excludes", "$remote_dns", "$ac_url", "$ac_retry_min",
                "$ac_retry_max", "$ac_timeout");
function proxy_update_profile(name) {}

function proxy_remove_profile(name) {}

function proxy_enable(name) {}

function proxy_disable() {}

function proxy_toggle() {}
interactive("proxy-toggle", "Toggle use of proxy.", proxy_toggle);

// Proxy modes.
PROXY_NONE        = 0;
PROXY_AUTO_DETECT = 1;
PROXY_AUTO_CONFIG = 2;
PROXY_SYSTEM      = 4;
PROXY_MANUAL      = 5;

function proxy_set_mode(mode) {}

function proxy_enable_mode_line(formatter) {}


** NOTES

* State

Changes made to the network.proxy.* preferences by any means other than this API must be noticed and
handled, otherwise its state will be inconsistent with Mozilla's. Therefore, network.proxy.* will be
watched and the situation will be handled in one of a few ways.

1. Force the pref to reset to the value defined by the current profile. Users would not be able to modify
proxy settings via about:config, for example - but neither would extensions be able to modify proxy settings.

2. Include a built-in proxy profile *Custom* which indicates that the proxy settings are no longer
controlled by this API.

3. Any other ideas?


* hooks

'enabled' is passed the profile name. 'disabled' has no arguments.

'profile_updated' is passed the profile name and exists for the sake of the mode line widget, which will
need to be notified when 'update_profile' is called on the currently-enabled profile.


* define and update functions

These accept proxy addresses in the form of 1.2.3.4 or 1.2.3.4:1234. If no port is specified, the
well-known port of 8080 is assumed for all proxies except SOCKS, which assumes 1080.

The '$ac_*' keywords correspond to Mozilla's autoconfig prefs.

Default values for all parameters correspond to Mozilla's default value, if any.


* toggle function

The implementation will remember the last profile used and toggle it when this command is called. If no
proxy was previously enabled, a message should be printed to the minibuffer saying so.


* enable_mode_line function

'formatter' is an optional argument which if present must be a function which accepts a single argument,
the name of the enabled proxy profile, or null if proxy is disabled, and returns the string printed to
the modeline.

The proxy widget's 'update' method should be added to proxy_*_hooks.
msg78 (view) Author: nicktastic Date: 2008-10-02.18:11:52
** Revision based on feedback

* API

define_hook("proxy_enabled_hook");
define_hook("proxy_disabled_hook");
define_hook("proxy_profile_updated_hook");

// Proxy modes.
PROXY_NONE        = 0;
PROXY_AUTO_DETECT = 1;
PROXY_AUTO_CONFIG = 2;
PROXY_SYSTEM      = 4;
PROXY_MANUAL      = 5;

define_keywords("$http", "$ssl", "$ftp", "$gopher", "$socks", "$socks_ver",
                "$socks_dns", "$excludes" "$ac_url", "$ac_retry_min",
                "$ac_retry_max", "$ac_timeout", "$mode");
function proxy_define_profile(name) {}

function proxy_remove_profile(name) {}

function proxy_enable(name) {}

function proxy_disable() {}

function proxy_toggle() {}
interactive("proxy-toggle", "Toggle use of proxy.", proxy_toggle);

function proxy_enable_mode_line(formatter) {}


* Changes

Eliminate 'proxy_update_profile' - 'proxy_define_profile' will handle updating existing profiles.

The proxy mode is now part of the profile.

retroj suggests having some pre-defined profiles for system and auto-detection. Profiles for popular
proxies like privoxy might not be a bad idea either.

Note that one would switch from one profile to another by calling 'proxy_profile_enable' with the new
profile name, without the need to first call 'proxy_disable'.
msg82 (view) Author: nicktastic Date: 2008-10-02.19:39:23
PROXY_NONE may be eliminated. Having a proxy profile which tells mozilla not to use a proxy is pointless.
msg83 (view) Author: jbms Date: 2008-10-02.23:48:22
It seems to me that having a single update function that takes a bunch of keyword arguments, only some of
which can be used together, is not optimal.  On the other hand, I can see that it is the easiest to
implement since it also maps most closely to the Mozilla prefs.  An alternative interface would be to
have separate functions for each proxy type, e.g. new http_proxy("whatever", $port = <optional port>)
or new socks_proxy("whatever", $port = whatever, $remote_dns)

or new auto_configuration_proxy(...);

It does certainly seem that disabling a proxy profile should be handled by switching to a special
no-proxy profile or something like that.  Or maybe this no-proxy profile would just be called null? 
Also, why does there need to be a special list of proxy profiles?  Why not have proxy profiles simply be
stored as variables?  They could have an optional label field for display purposes.

As far as something other than the proxy mechanism changing the profile, I don't think the proxy profile
system should try to prevent the preferences from being changed by something else.  Instead, as you
suggest as one possibility, it can just add a watch on all of the proxy prefs.  If the proxy prefs change
under it in a way that matters (changes on prefs that don't have any effect due to other prefs can just
be ignored), it can either set the stored current profile to the no-proxy profile if that prefs indicate
that a proxy is disabled, or alternatively it can create a new un-named profile corresponding to the
current prefs.
msg88 (view) Author: nicktastic Date: 2008-10-08.18:33:00
> It seems to me that having a single update function that takes a bunch of keyword arguments, only some of
> which can be used together, is not optimal.  On the other hand, I can see that it is the easiest to
> implement since it also maps most closely to the Mozilla prefs.  An alternative interface would be to
> have separate functions for each proxy type, e.g. new http_proxy("whatever", $port = <optional port>)
> or new socks_proxy("whatever", $port = whatever, $remote_dns)
>
> or new auto_configuration_proxy(...);

Most of the arguments can be used together. If a SOCKS proxy is
specified it will be used unless a protocol-specific proxy is also
specified. So if I specify only HTTP and SOCKS proxies, HTTP requests
will use the former, and SSL, FTP, and gopher will use the latter. The
$ac_* and $mode parameters are the ones that don't quite fit, which is
one reason why they weren't part of the profile in the first version.
However I believe it makes more sense to include them as part of a
profile.

> It does certainly seem that disabling a proxy profile should be handled by switching to a special
> no-proxy profile or something like that.  Or maybe this no-proxy profile would just be called null?

Are you suggesting that proxy_disable() be removed and instead using
e.g. proxy_enable(null) to disable the active proxy? I think using a
function named 'enable' to disable is misleading. Having
proxy_enable(null) disable active proxy is fine, but I think
proxy_disable() should remain, even though one will just call the
other.

> Also, why does there need to be a special list of proxy profiles?  Why not have proxy profiles simply be
> stored as variables?  They could have an optional label field for display purposes.

Internally I envisioned using a proxy_profile object whose properties
were the keywords accepted by 'define', and these would be stored in a
list with contents manipulated by 'define' and 'remove'. Expecting
users to manage these objects themselves would give them more control
and eliminate the need for 'define' and 'remove' thus simplifying the
API. However, I can't imagine any especial gain by doing so
(suggestions welcome), while I can imagine a scenario where doing so
would be detrimental: interactive profile configuration and management
systems.

For example, an interactive command 'proxy-enable', which I hadn't
considered before, but which now strikes me as useful. The alternative
is having users edit their rc files and restart/reinit conkeror, or to
eval javascript, everytime they want to change proxies. If a user has
a proxy on-campus but not at home, and uses conkeror on his laptop at
both locations, it is much more reasonable to have him type into
conkeror:

M-x proxy-enable<return>Campus<return>

...than to have him edit his rc file when he gets to school, and then
again when he gets home.

For another example, someday we may decide to include interactive
profile configuration for users who are uncomfortable configuring a
browser via an rc file.

These examples require a managed list of profiles. Per the first
example, 'proxy-enable' would need to know where to look to find the
profile named 'Campus', which it couldn't do if the user stores his
profiles in some unknown variable, list, map, or object.

Sure, we could merge approaches: keep 'define' and 'remove'
specifically for convenience and for use by interactive systems, and
if the argument to proxy_enable is a string, then consult the managed
list, and if it is a proxy_profile instance, then just use it. Though
I believe this route would confuse things for little benefit, I also
enjoy the liberty of having the best of both worlds.

What do you think?

> As far as something other than the proxy mechanism changing the profile, I don't think the proxy profile
> < ... snip ... >
> current prefs.

Agreed.
msg89 (view) Author: jbms Date: 2008-10-08.18:51:39
"Nicholas A. Zigarovich" <nicktastic@gmail.com> writes:

> Most of the arguments can be used together. If a SOCKS proxy is
> specified it will be used unless a protocol-specific proxy is also
> specified. So if I specify only HTTP and SOCKS proxies, HTTP requests
> will use the former, and SSL, FTP, and gopher will use the latter.

Okay, I see.  It certainly makes sense then to include all of those
parameters together.

> The $ac_* and $mode parameters are the ones that don't quite fit,
> which is one reason why they weren't part of the profile in the first
> version.  However I believe it makes more sense to include them as
> part of a profile.

Perhaps then the split should just be between a regular proxy profile
and an auto-configuration proxy profile.

>> It does certainly seem that disabling a proxy profile should be handled by
> switching to a special
>> no-proxy profile or something like that.  Or maybe this no-proxy profile would
> just be called null?

> Are you suggesting that proxy_disable() be removed and instead using
> e.g. proxy_enable(null) to disable the active proxy? I think using a
> function named 'enable' to disable is misleading. Having
> proxy_enable(null) disable active proxy is fine, but I think
> proxy_disable() should remain, even though one will just call the
> other.

Well, it could be called proxy_switch_profile, but proxy_disable is fine
as well, I suppose.

>> Also, why does there need to be a special list of proxy profiles?  Why not
> have proxy profiles simply be
>> stored as variables?  They could have an optional label field for display
> purposes.

> Internally I envisioned using a proxy_profile object whose properties
> were the keywords accepted by 'define', and these would be stored in a
> list with contents manipulated by 'define' and 'remove'. Expecting
> users to manage these objects themselves would give them more control
> and eliminate the need for 'define' and 'remove' thus simplifying the
> API. However, I can't imagine any especial gain by doing so
> (suggestions welcome), while I can imagine a scenario where doing so
> would be detrimental: interactive profile configuration and management
> systems.

> For example, an interactive command 'proxy-enable', which I hadn't
> considered before, but which now strikes me as useful. The alternative
> is having users edit their rc files and restart/reinit conkeror, or to
> eval javascript, everytime they want to change proxies. If a user has
> a proxy on-campus but not at home, and uses conkeror on his laptop at
> both locations, it is much more reasonable to have him type into
> conkeror:

> M-x proxy-enable<return>Campus<return>

> ...than to have him edit his rc file when he gets to school, and then
> again when he gets home.

Well, for this purpose, I would think the user would want some more
general system that automatically handles all of his programs, and
ideally auto-detects whether he is on campus, since I would think it
would be a real pain to have to do some manual configuration of every
program.

Still, I can see providing an interactive command for proxy switching
with completion a useful thing, so I'll agree that having a special list
is a good idea.
msg95 (view) Author: nicktastic Date: 2008-10-10.18:29:59
Regarding the problem of splitting off the $mode and $ac_* keywords, I've come up with a few solutions,
but the following is my favorite.

    const PROXY_AUTO_DETECT = new proxy_profile("*Auto-Detect*");
    const PROXY_AUTO_CONFIG = new proxy_profile("*Auto-Config*");
    const PROXY_AUTO_SYSTEM = new proxy_profile("*System*");

    keywords("$retry_min", "$retry_max", "$timeout");
    function proxy_auto_config(url) {}

To auto-detect or to use the system proxy, the user just passes the appropriate const profile to
proxy_enable since no arguments are required for these types. To auto-config, call proxy_auto_config()
passing the auto-config url.

This solution works well with the profile hooks as defined thus far, and also with the mode line, which
is expected to use those hooks.
msg98 (view) Author: nicktastic Date: 2008-10-10.22:20:57
New plan. First, get proxy profiles working. Next, consider an API suitable for use by interactive
systems. The old define/remove system really doesn't work well with profile classes.

Barring any further comments or concerns, this will be the last revision before I start implementing.

{
    define_hook("proxy_switched_hook");

    function proxy_profile() {}
    proxy_profile.prototype = {};

    define_keywords("$http", "$ssl", "$ftp", "$gopher", "$socks", "$socks_ver",
                    "$socks_dns", "$excludes");
    function proxy_profile_manual(name) {}
    proxy_profile_manual.prototype = new proxy_profile();
    proxy_profile_manual.prototype.name = null;
    proxy_profile_manual.prototype.http = null;
    proxy_profile_manual.prototype.ssl = null;
    proxy_profile_manual.prototype.ftp = null;
    proxy_profile_manual.prototype.gopher = null;
    proxy_profile_manual.prototype.socks = null;
    proxy_profile_manual.prototype.socks_ver = null;
    proxy_profile_manual.prototype.socks_dns = null;

    function proxy_profile_auto_detect() {}
    proxy_profile_auto_detect.prototype = new proxy_profile();

    keywords("$retry_min", "$retry_max", "$timeout");
    function proxy_profile_auto_config(name, url) {}
    proxy_profile_auto_config.prototype = new proxy_profile();
    proxy_profile_auto_config.prototype.name = null;
    proxy_profile_auto_config.prototype.url = null;
    proxy_profile_auto_config.prototype.retry_min = null;
    proxy_profile_auto_config.prototype.retry_max = null;
    proxy_profile_auto_config.prototype.timeout = null;

    function proxy_profile_system() {}
    proxy_profile_system.prototype = new proxy_profile();

    function proxy_profile_null() {}
    proxy_profile_null.prototype = new proxy_profile();

    function proxy_profile_unmanaged() {}
    proxy_profile_unmanaged.prototype = new proxy_profile();

    let toggle_profile = null;

    function proxy_switch(profile) {}

    function proxy_toggle() {}
    interactive("proxy-toggle", "Toggle use of active profile.", function (I) {});

    function proxy_mode_line_formatter_default(profile) {}

    function proxy_mode_line_enable(formatter) {}
}
msg435 (view) Author: XTaran Date: 2009-10-05.00:07:39
I would be very happy if there would be an interface to change proxy settings. Simple profiles with
switching between them seems to be the simplest and easiest to implement idea.

I'm also fine with a more complex solution, but having a simple but not idiot proof solution instead of
nothing like now, would be a great start.

I though prefer the simpler (and IMHO straight forward) solution anyway, but won't stop anyone in
implementing the later suggestion. :-)
msg436 (view) Author: retroj Date: 2009-10-05.19:41:12
Discussion on irc has brought us back to the idea of starting with the api
outlined in msg78, but with some changes.  First of all, there does need
to be a global list of profiles, because there is a demand for being able
to enable, disable, and switch profiles via interactive commands.

The procedures to define and undefine profiles should be called
define_proxy_profile and undefine_proxy_profile in order to be consistent
with existing conventions.  Note that the `undefine' procedure is probably
not necessary, but we might as well have it for api completeness, given
that it is trivial to write.

PROXY_NONE is not useless, as it says in msg82.. or at least it is the
case that poeple will want to be able to interactively disable their
proxy.

Last is the question of the exact call form of define_proxy_profile.  In
my view, the best would be:

   function define_proxy_profile (name, obj) {}


  ..where OBJ is an ordinary javascript object possibly containing a field
for each of the Mozilla proxy prefs.  This object can be used directly as
the profile value stored in the global list.  If for whatever reason we
need to add properties, methods, or accesssors to it, that can be done
concisely by setting its __proto__.  Though I happily defend the use of
keywords as syntactic sugar in many cases, here I don't think they are the
right tool for the job, as they impart no benefit, and a simple javascript
object is the obvious, simplest implementation.  Additionally:

 * The keywords syntactic sugar is best suited to conveying "extra"
   information to a procedure or constructor, but in the case of
   define_proxy_profile, all of the fields are "main" information.

 * By convention, `define*' procedures that assign a value to something
   have a form like:

     define_thing(name, value);

   This has a more obvious meaning than the form:

     define_thing(name, $option, $option, $option);
msg437 (view) Author: retroj Date: 2009-10-05.20:19:36
Forgot to mention in my previous post..

Having subclasses for certain proxy profiles as described in msg83 could possibly be convenient,
and the system described in my previous post would be forward-compatible with such a system.  As
a hypothetical example, a call to define_proxy_profile might look like this:

   define_proxy_profile("myproxy", new http_proxy(1080));

Note that it is not necessary to implement such subclasses for the functioning of the system, and
they only provide a kind of shorthand notation.  But if we leave the possibility open, we may decide
later on that they are a feature worth having.
History
Date User Action Args
2009-10-05 20:19:36retrojsetmessages: + msg437
2009-10-05 19:41:12retrojsetmessages: + msg436
2009-10-05 00:07:39XTaransetmessages: + msg435
2008-10-10 22:21:08nicktasticsetmessages: - msg97
2008-10-10 22:20:57nicktasticsetmessages: + msg98
2008-10-10 20:10:52nicktasticsetmessages: - msg96
2008-10-10 20:10:03nicktasticsetmessages: + msg97
2008-10-10 19:42:10nicktasticsetmessages: + msg96
2008-10-10 18:29:59nicktasticsetmessages: + msg95
2008-10-08 19:52:03nicktasticsetpriority: bug -> wish
2008-10-08 18:51:39jbmssetpriority: wish -> bug
messages: + msg89
2008-10-08 18:41:57nicktasticsetpriority: bug -> wish
2008-10-08 18:33:00nicktasticsetpriority: wish -> bug
messages: + msg88
2008-10-02 23:48:22jbmssetmessages: + msg83
2008-10-02 19:39:23nicktasticsetmessages: + msg82
2008-10-02 18:11:52nicktasticsetstatus: unread -> chatting
messages: + msg78
2008-10-02 17:34:25nicktasticcreate