From owner-ietf-openproxy@mail.imc.org Mon Feb 3 10:48:08 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA10993 for ; Mon, 3 Feb 2003 10:48:07 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h13FOAl26969 for ietf-openproxy-bks; Mon, 3 Feb 2003 07:24:10 -0800 (PST) Received: from dirty.research.bell-labs.com (dirty.research.bell-labs.com [204.178.16.6]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h13FO9d26962 for ; Mon, 3 Feb 2003 07:24:09 -0800 (PST) Received: from scummy.research.bell-labs.com (H-135-104-2-10.research.bell-labs.com [135.104.2.10]) by dirty.research.bell-labs.com (8.12.5/8.12.5) with ESMTP id h13FOAhN010092 for ; Mon, 3 Feb 2003 10:24:10 -0500 (EST) Received: from bronx.dnrc.bell-labs.com (bronx.dnrc.bell-labs.com [135.180.160.8]) by scummy.research.bell-labs.com (8.11.6/8.11.6) with ESMTP id h13FNwI41642 for ; Mon, 3 Feb 2003 10:23:59 -0500 (EST) Received: from mhof.com (biena [135.180.160.86]) by bronx.dnrc.bell-labs.com (8.9.3/8.9.3) with ESMTP id KAA19268 for ; Mon, 3 Feb 2003 10:23:58 -0500 (EST) Message-ID: <3E3E89AD.3070403@mhof.com> Date: Mon, 03 Feb 2003 10:24:29 -0500 From: Markus Hofmann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.2.1) Gecko/20021130 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "'OPES Group'" Subject: Re: FW: publish draft: draft-ietf-opes-authorization-01 References: <87609AFB433BD5118D5E0002A52CD75404B3C813@zcard0k6.ca.nortel.com> <3E37F73E.4060209@cs.wayne.edu> In-Reply-To: <3E37F73E.4060209@cs.wayne.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit Weisong Shi wrote: > I don't think the notion of group and membership is a good idea > here. Sometime, it is incorrect. For example, let assume user Alice > wants service 1 and service 2, and in the order 1,2. user Bob wants > service 2 and service 1 too, but in the order 2,1. In this case, > without calling the service for each request send by either Alice or > Bob, we can not guarantee the correcness of service execution. In this particular case, you could specify two separate groups - the first one includes users who 'subscribed' to executing service 1 followed by service 2, and a second group of users who 'subscribed' to executing the two services in reverse order. The notion of groups is still beneficial, since there can be multiple users in both groups. -Markus From owner-ietf-openproxy@mail.imc.org Thu Feb 6 14:25:56 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA01059 for ; Thu, 6 Feb 2003 14:25:55 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h16J0ax25774 for ietf-openproxy-bks; Thu, 6 Feb 2003 11:00:36 -0800 (PST) Received: from mauve.mrochek.com (mauve.mrochek.com [209.55.107.55]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h16J0Zd25769 for ; Thu, 6 Feb 2003 11:00:35 -0800 (PST) Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01KS40VACSLS002O3W@mauve.mrochek.com> for ietf-openproxy@imc.org; Thu, 06 Feb 2003 11:00:35 -0800 (PST) Date: Thu, 06 Feb 2003 10:59:05 -0800 (PST) From: ned.freed@mrochek.com Subject: IESG concerns regarding the OPES requirements and architecture documents To: ietf-openproxy@imc.org Cc: ned.freed@mrochek.com, paf@cisco.com, "Marshall T. Rose" , Markus Hofmann Message-id: <01KS45U12KSW002O3W@mauve.mrochek.com> MIME-version: 1.0 Content-type: TEXT/PLAIN; CHARSET=us-ascii Content-transfer-encoding: 7BIT Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7BIT The IESG has approved the drafts ietf-opes-protocol-reqs-03.txt draft-ietf-opes-architecture-04.txt for publication as informational RFCs. An announcement to this effect will be sent shortly. As explained previously, the IESG believes that draft-ietf-opes-scenarios-01.txt is too dependent on draft-ietf-opes-threats-00.txt to publish at this time. The IESG is concerned, however, that what appears in these drafts is a fairly high level overview of OPES requirements and architecture. Missing is any mention of specific mechanisms to provide security and congestions. This is not necessarily a problem given these are informational requirement and architecture documents. However, the IESG notes that the lack of specificity at this point could lead to problems deciding on actual protocol details later in the process. Requirement and architecture documents have often proved to be useful tools in reaching consensus on subsequent protocol specifications. Should difficulties arise in reaching consensus later it may be necessary and useful to expand on these documents with information about specific mechanisms to be used. Ned Freed, for the IESG From owner-ietf-openproxy@mail.imc.org Mon Feb 10 13:15:59 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA19021 for ; Mon, 10 Feb 2003 13:15:53 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1AHafF09843 for ietf-openproxy-bks; Mon, 10 Feb 2003 09:36:41 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1AHadd09838 for ; Mon, 10 Feb 2003 09:36:40 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1AHa8428068; Mon, 10 Feb 2003 12:36:08 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <1LM310VW>; Mon, 10 Feb 2003 12:36:08 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404D51543@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: 'Internet-Drafts@ietf.org' Cc: "'Markus Hofmann'" , "'Marshall Rose'" , "'OPES Group'" , "Abbie Barbir" Subject: publish draft: draft-ietf-opes-authorization-02 Date: Mon, 10 Feb 2003 12:36:07 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C2D12A.E490426A" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C2D12A.E490426A Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D12A.E490426A" ------_=_NextPart_001_01C2D12A.E490426A Content-Type: text/plain Please publish the atatched OPES WG document as draft-ietf-opes-authorization-02. thanks Abbie Barbir Nortel Networks ------_=_NextPart_001_01C2D12A.E490426A Content-Type: text/html publish draft: draft-ietf-opes-authorization-02

Please publish the atatched OPES WG document as draft-ietf-opes-authorization-02.

thanks


Abbie Barbir
Nortel Networks

  ------_=_NextPart_001_01C2D12A.E490426A-- ------_=_NextPart_000_01C2D12A.E490426A Content-Type: text/plain; name="draft-ietf-opes-authorization-02.txt" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="draft-ietf-opes-authorization-02.txt" Content-Transfer-Encoding: quoted-printable Network Working Group A. = Barbir Internet-Draft Nortel = Networks Expires: August 11, 2003 O. = Batuner = Consultant A. = Beck Lucent = Technologies T. = Chan = Nokia H. = Orman Purple Streak = Development February 10, = 2003 Policy, Authorization and Enforcement Requirements of OPES draft-ietf-opes-authorization-02 Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that = other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six = months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on August 11, 2003. Copyright Notice Copyright (C) The Internet Society (2003). All Rights Reserved. Abstract This document describes policy, authorization and enforcement requirements for the selection of the services to be applied to a given OPES flow. Barbir, et al. Expires August 11, 2003 [Page = 1] =0C Internet-Draft Policy Requirements February = 2003 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . = 3 2. Policy Architecture . . . . . . . . . . . . . . . . . . . . = 4 2.1 Policy Components and Functions . . . . . . . . . . . . . . = 4 2.2 Requirements For Policy Decision Point . . . . . . . . . . . = 6 2.3 Requirements for Policy Enforcement Points . . . . . . . . . = 6 3. Requirements for Interfaces . . . . . . . . . . . . . . . . = 8 3.1 Service Bindings Requirements . . . . . . . . . . . . . . . = 8 3.1.1 Environment Variables . . . . . . . . . . . . . . . . . . . = 8 3.1.2 Requirements for Using State Information . . . . . . . . . . = 9 3.1.3 Requirements for Passing Information Between Services . . . = 9 3.2 Requirements for Rule and Rules Management . . . . . . . . . = 9 3.2.1 Requirements for Rule Providers . . . . . . . . . . . . . . = 9 3.2.2 Requirements for Rule Formats and Protocols . . . . . . . . = 10 3.2.3 Requirements for Rule Conditions . . . . . . . . . . . . . . = 10 3.2.4 Requirements for Rule Actions . . . . . . . . . . . . . . . = 10 3.3 Requirements for Policy Expression . . . . . . . . . . . . . = 11 4. Authentication of Principals and Authorization of Services . . . . . . . . . . . . . . . . . . . . . . . . . . = 12 4.1 End users, Publishers and Other Considerations . . . . . . . = 12 4.1.1 Considerations for end users . . . . . . . . . . . . . . . . = 12 4.1.2 Considerations for publishing sites . . . . . . . . . . . . = 13 4.1.3 Other considerations . . . . . . . . . . . . . . . . . . . . = 13 4.2 Authentication . . . . . . . . . . . . . . . . . . . . . . . = 13 4.3 Authorization . . . . . . . . . . . . . . . . . . . . . . . = 14 4.4 Integrity and Encryption . . . . . . . . . . . . . . . . . . = 15 4.4.1 Integrity and confidentiality of authentication and requests/responses for service . . . . . . . . . . . . . . . = 15 4.4.2 Integrity and confidentiality of application content . . . . = 15 4.5 Privacy . . . . . . . . . . . . . . . . . . . . . . . . . . = 16 Normative References . . . . . . . . . . . . . . . . . . . . = 17 Informative References . . . . . . . . . . . . . . . . . . . = 18 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . = 18 A. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . = 20 Intellectual Property and Copyright Statements . . . . . . . = 21 Barbir, et al. Expires August 11, 2003 [Page = 2] =0C Internet-Draft Policy Requirements February = 2003 1. Introduction The Open Pluggable Edge Services (OPES) [1] architecture enables cooperative application services (OPES services) between a data provider, a data consumer, and zero or more OPES processors. The application services under consideration analyze and possibly transform application-level messages exchanged between the data provider and the data consumer. The OPES processor can distribute the responsibility of service execution by communicating and collaborating with one or more remote callout servers. The execution of such services is governed by a set of rules installed on OPES processor. The rule evaluation can trigger the execution of service applications local to the OPES processor or on = a remote callout server. Policies express the goals of an OPES processor as a set of rules used to administer, manage and control access to resources. The requirements in this document govern the behavior of OPES entities = in determining which, if any, of available services are to be applied = to a given message. The scope of OPES policies described in this document are limited to those that describe which services to call and, if appropriate, with what parameters. These policies do not include those that prescribe the behavior of the called services. It is desirable to enable a common management framework for specifying policies for both the calling of and the behavior of a service. The integration of such function is the domain of policy administration user interaction applications. The document is organized as follows:Section 2 considers policy framework. Section 3 discusses requirements for interfaces, while section 4 examines authentication of principals and authorization of services. Barbir, et al. Expires August 11, 2003 [Page = 3] =0C Internet-Draft Policy Requirements February = 2003 2. Policy Architecture This section describes the architectural policy decomposition requirements. It also describes the requirements for the interfaces between the policy components. 2.1 Policy Components and Functions The policy functions are decomposed into three components: a Rule Author, a Policy Decision Point (PDP) and Policy Enforcement Point (PEP). The Rule Author provides the rules to be used by an OPES entity. These rules control the invocation of services on behalf of the rule author. The PDP and the PEP interpret the collected rules and appropriately enforce them. The decomposition is illustrated in Figure 1. Barbir, et al. Expires August 11, 2003 [Page = 4] =0C Internet-Draft Policy Requirements February = 2003 +--------+ +--------+ | Rule | | Rule | | Author | ... | Author | +--------+ +--------+ | | | | | +----------+ | | | Policy | | <- PDP Interface +--------->| Decision |<----------+ | Point | +----------+ | ^ | | | | <- PEP Interface | | V | +--------------+ ... ---> | Policy | ---> | Enforcement | Data Traffic <--- | Point | <--- +--------------+ Figure 1: Policy Components The decomposition of policy control into a PDP and a PEP permit the offloading of some tasks to an administrative service that may be located on a separate server from the real-time enforcement services of the PEP that reside on the OPES processor. The PDP provides for the authentication and authorization of rule authors and the validation and compilation of rules. The PEP resides in the data filter where the data from an OPES flow is evaluated against the compiled rules and appropriate calls to the requested services are performed. Interfaces between these architectural components are points of interoperability. The interface between rule authors and the policy decision points (PDP Interface) must use the standard format that = may result from the requirements as described in this document. The interface between the policy decision points and the policy enforcement points (PEP Interface) can be internal to a specific vendor implementation of an OPES processor. Implementations must use standard interface only if the PDP and the PEP reside on different OPES processor. Barbir, et al. Expires August 11, 2003 [Page = 5] =0C Internet-Draft Policy Requirements February = 2003 2.2 Requirements For Policy Decision Point The Policy Decision Point is essentially a policy compiler. The PDP must be a service that provides administrative support to the enforcement points. The PDP service must authenticate the rule authors. The PDP must verify that the specified rules are within the scope of the rule authors authority. The PDP must be a component of the OPES Administration Authority. 2.3 Requirements for Policy Enforcement Points In the OPES architecture, the data filter represents a Policy Enforcement point (PEP). At this point, data from an OPES flow is evaluated against the compiled rules and appropriate calls to the requested services are performed. In the PEP rules may chain actions together, where, a series of services to be called are specified. Implementation must ensure the passing of information from one called service to another. Implementation must not prohibit the re-evaluation of a message to determine if another service or set of services should be called. The execution of an action (i.e., the triggering of a rule) may lead to the modification of a message property values. For example, an OPES service that under some circumstances converts JPEG images to GIF images modifies the content type of the requested web object. Such modification of message property values may change the behavior of subsequently performed OPES actions. The data filter should act = on matched rules before it evaluates subsequent rules. Multiple = matched rules can be triggered simultaneously if the data filter can determine in advance that there are no side effects from the execution of any specific rule. A data filter may evaluate messages several times in the course of handling an OPES flow. The rule processing points may be defined by administratively defined names. The definition of such names can serve as a selector for policy rules to determine the applicability of a rule or a set of rules at each processing point. The scope of policy control of policy roles as defined RFC 3060 should be used where it aids in the development of the OPES policy model. In Figure 2 a typical message data flow between a data consumer application, an OPES processor and a data provider application. = There are four commonly used processing points identified by the numbers 1 through 4. Barbir, et al. Expires August 11, 2003 [Page = 6] =0C Internet-Draft Policy Requirements February = 2003 +--------+ +-----------+ +---------+ | |<------|4 3|<------| | | Data | | OPES | | Data | |Consumer| | Processor | |Provider | | Appl. |------>|1 2|------>| Appl. | +--------+ +-----------+ +---------+ Figure 2: Processing Execution Points Any data filter (PEP) or any administrative (PDP) implementation = must support the four rule processing points. o Data Consumer Request Handling Role : This involves request processing when received from a Data Consumer Application. o OPES Processor Request handling role: This involves request processing before forwarding to Data Provider Application. o Data Provider Response handling role: This involves response processing when forwarding to Data Consumer Application. o OPES Processor Response handling role:This involves response processing when forwarding to Data Consumer Application. Barbir, et al. Expires August 11, 2003 [Page = 7] =0C Internet-Draft Policy Requirements February = 2003 3. Requirements for Interfaces The interface between the policy system and OPES services needs to include the ability to pass system state information as well as the subject message. 3.1 Service Bindings Requirements The invoked OPES services must be able to be specified in a location independent fashion. That is, the rule authors need not know and = need not specify the instance of an OPES service in the rules. The rule author should be able to identify the required service at the detail level that is appropriate for his or her needs. The rule author should be able to specify a type of service or be able to specify any service that fits a general category of service to be applied its traffic. The binding of OPES service names to specific service may be distributed between the PDP and the PEP. As rules are compiled and validated by the PDP, they must be resolved to a specific installations' set of homogeneous OPES service. The selection of a specific instance may be postponed and left to = PEP to select at either rule installation time or at run time. To = achieve interoperability, PEP must support resolving a generic name to a specific instance. It is possible to use services such as SLP or = UDDI to resolve generic service names to specific OPES service instances. The policy system may support dynamic discovery of service bindings. The rule author may, not know specific service bindings such as protocol and parameters, when a rule (as specified on the PDP Interface) is general in nature. The required binding information must be provided by the PDP and conveyed on the PEP Interface. A service description methodology such as WSDL must be present in the policy system. It is to be determined whether an OPES standard is required. 3.1.1 Environment Variables There may be a need to define and support means for maintaining = state information that can be used in both condition evaluation and action execution. Depending on the execution environment, OPES services may have the freedom to define variables that are needed and use these variables to further define their service behavior without the = data filter support. Barbir, et al. Expires August 11, 2003 [Page = 8] =0C Internet-Draft Policy Requirements February = 2003 3.1.2 Requirements for Using State Information Policy rules may specify that state information be used as part of the evaluation of the rules against a given message in an OPES flow. Thus, the policy system should support the maintenance of groups = that can be used in evaluating rule conditions. Membership in such = groups can be used as action triggers. For example, an authorized site blocking service might conclude that a particular user shouldn't be permitted access to a certain web site. Rather than calling the service for each request sent by such = a user, a rule might be created that determine if user is a member of blocked users and requested site is a member of blocked-sites then invoke a local blocking service to return and return an appropriate message to the user. 3.1.3 Requirements for Passing Information Between Services Environment variables can be used to pass state information between services. For example, analysis of the request or modifications to the request may need to be captured as state information that can be passed to other services on the request path or to services on the response(s) associated with that request. In the PEP, there should be provisions to enable setting up = variables when returning from a service call and passing variables to other called services based on policy. 3.2 Requirements for Rule and Rules Management This section provides the requirements for rule management. The = rules are divided into two groups. Some rules are provided by the data consumer application and other rules are provided by the data provider application. 3.2.1 Requirements for Rule Providers The requirements for rule providers are: o Rule providers must be authenticated and authorized for rules = that apply to their network role. o Rule providers must not be able to specify rules that are not within their scope of authority. o Rule providers should be able to specify only what is needed for their services. Barbir, et al. Expires August 11, 2003 [Page = 9] =0C Internet-Draft Policy Requirements February = 2003 o Compilation of rules from different sources must not lead to execution of conflicting rules. o The resolution of such rule conflicts is out of scope o Rules are assumed to be static and applied to current network state. 3.2.2 Requirements for Rule Formats and Protocols It is desirable to choose standard technologies like XML to specify the rule language format. Rules need to be sent form the rule authors to the OPES administrative server for service authorization, rule validation and compilation. The mechanisms for doing that are out of scope of the current work. Once the rules are authorized, validated and compiled by the administrative server, the rules need to be sent to the OPES processor. The mechanisms for doing that are out of scope of the current work. 3.2.3 Requirements for Rule Conditions Rule conditions must be matched against attribute values of the encapsulated protocol as well as environment variable values. Attribute values of the encapsulated protocol include protocol = header values and possibly also protocol body values. Some OPES services may need to be invoked for all users requests or server responses, services with logging functionality, as an = example. The rule system should allow unconditional rules rather than requiring rule authors to specify rule conditions that are always true. 3.2.4 Requirements for Rule Actions The rule system must allow for the specification of rule actions = that are triggered if the conditions of a rule are met. Matched rules typically lead to the invocation of local or remote services. Rule actions must identify the OPES service that is to be executed for = the current message request or response. Rule actions may contain run-time parameters which can be used to control the behavior of an OPES service. If specified, these parameters must be passed to the executed OPES service. Barbir, et al. Expires August 11, 2003 [Page = 10] =0C Internet-Draft Policy Requirements February = 2003 3.3 Requirements for Policy Expression OPES processors must enforce policy requirements set by data consumers and/or data publishers in accordance with the architecture [ref ARCH] and this document. They cannot do this consistently unless there is an unambiguous semantics and representation of the data elements mentioned in the policy. For example, this document mentions protection of user "identity" and "profile" information. = If a user specifies that his identity must not be shared with other = OPES administrative trust domains and later discovers that his family = name has been shared, he might complain. If he were told that "family names are not considered 'identities' by this site", he would probably feel that he had cause for complaint. Or, he might be told that when he selected "do not share identity" on a web form offered by the OPES service provider, that this only covered his login name, and that a different part of the form had to be filled out to = protect family name. A further breakdown can occur if the configuration information provided by such a web form gets translated into configuration elements given to an OPES processor, and those configuration elements are difficult for a software engineer to translate into policy enforcement. The data elements might have confusing names or be split into groupings that are difficult to relate to one another. The examples illustrate why OPES policy must have definitions of = data elements, their relationships, and how they relate to enforcement. These semantics of essential items do not require a separate protocol, but they must be agreed upon by all OPES service = providers, and the users of OPES services must be assured that they have the ability to know their settings, to change them if the service provider policy allows the changes, and to have reasonable assurance that they are enforced with reasonable interpretations. The requirements for policy data elements in the OPES specification do not have to be all-inclusive, but they must cover the minimal set of elements that enable the policies that protect the data of end users and publishers. Barbir, et al. Expires August 11, 2003 [Page = 11] =0C Internet-Draft Policy Requirements February = 2003 4. Authentication of Principals and Authorization of Services This section considers the authorization and authentication of OPES services. 4.1 End users, Publishers and Other Considerations 4.1.1 Considerations for end users An OPES rule determines which attributes of traffic will trigger the application of an OPES services. The author of the service can supply rules, but the author cannot supply the necessary part of the rule precondition that determines which network users will have the OPES services applied for them. This section discusses how users = are identified in the rule preconditions, and how users can select and deselect OPES services for their traffic, how an OPES service provider should identify the users, and how they determine whether = or not to add their service selection to an OPES enforcement point. An OPES service provider must satisfy these major requirements: o Allow all users to request addition, deletion, or blocking of = OPES services for their traffic (blocking means "do not use this service for my traffic"). o Prevent untrusted users from causing OPES services to interfere with the traffic of other users. o Allow users to see their OPES service profiles and notify them of changes. o Keep a log of all profile activity for audit purposes. o Adhere to a privacy policy guarding users' profiles. The administrator of the PDP is a trusted party and can set policy for individuals or groups using out-of-band communication and configuration files. However, users must always be able to query = the PDP in order to learn what rules apply to their traffic. Rules can be deposited in the PDP with no precondition relating to network users. This is the way rules are packaged with an OPES service when it is delivered for installation. The PDP is responsible for binding identities to the rules and transmitting = them to the PEP. The identity used by the PDP for policy decisions must = be strictly mapped to the identity used by the PEP. Thus, if a user goes through and identification and authentication procedure with = the PDP and is known by identity "A", and if the PEP uses IP addresses Barbir, et al. Expires August 11, 2003 [Page = 12] =0C Internet-Draft Policy Requirements February = 2003 for identities, then the PDP must provide the PEP with a binding between "A" and A's current IP address. 4.1.2 Considerations for publishing sites An OPES service provider acting on behalf of different publishing sites should keep all the above considerations in mind when implementing an OPES site. Because each publishing site may be represented by only a single identity, the authentication and authorization databases may be easier for the PEP to handle. 4.1.3 Other considerations Authentication may be necessary between PDP's and PEP's, PEP's and callout servers, PEP's and other PEP's, callout servers and other callout servers, for purposes of validating privacy policies. In = any case where user data or traffic crosses trust domain boundaries, the originating trust domain should have a policy describing which other domains are trusted, and it should authenticate the domains and = their policies before forwarding information. 4.2 Authentication When an individual selects (or deselects) an OPES service, the individual must be authenticated by the OPES service provider. This means that a binding between the user's communication channel and an identity known to the service provider is made in a secure manner. This SHOULD be done using a strong authentication method with a public key certificate for the user; this will be helpful in resolving later disputes. It is recommended that the service provider keep a log of all requests for OPES services. The service provider SHOULD use public key certficates to authenticate responses to requests. The service provider may have trusted users who through explicit or implicit contract can assign, remove, or block OPES services for particular users. The trusted users MUST be authenticated before being allowed to take actions which will modify the policy base, and thus, the actions of the PEP's. Because of the sensitivity of user profiles, the PEP Interface between the PEP and the PDP MUST use a secure transport protocol. = The PEP's must adhere to the privacy preferences of the users. When an OPES service provider accepts an OPES service, there must be a unique name for the service provided by the entity publishing the service. Users may refer to the unique name when requesting a service. The unique name must be when notifying users about their Barbir, et al. Expires August 11, 2003 [Page = 13] =0C Internet-Draft Policy Requirements February = 2003 service profiles. PEP's must be aware of the unique name for each service that can be accessed from their domain. There MUST be a cryptographic binding between the unique name and the entity responsible for the functional behavior of the service; i.e., if it is a human language translating service then the name of company = that wrote the software should be bound to the unique name. 4.3 Authorization In addition to requesting or terminating specific services, users = may block particular services, indicating that the services should not = be applied to their traffic. The "block all OPES" directive must be supported on a per user basis. A response to a request for an OPES service can be positive or negative. Reasons for a negative response include "service unknown" or "service denied by PDP policy". Positive responses should = include the identity of the requestor and the service and the type of request. As described in the OPES Architecture [1], requests for OPES = services originate in either the enduser or the publisher domain. The PDP bases its authorization decision on the requestor and the domain. There are some cases where the decision may be complicated. o The end user has blocked a service, but a trusted user of the PDP wants it applied anyway. In this case, the end user SHOULD prevail, unless their are security or legal reasons to leave it = in place. o The publisher and the enduser are in the same domain. If the publisher and enduser are both clients of a PDP, can they make requests that effect each other's processing? In this case, the PDP must have policy rules naming the identities that are allowed to set such rules. o The publisher requests a service for an enduser. In this case, = in which the PDP and PEP are in the publisher's administrative domain, the publisher has some way of identifying the end user = and his traffic, and the PDP must enable the PEP to enforce the policy. This is allowed, but the PDP MUST use strong methods to identify the user and his traffic. The user must be able to request and receive information about the service profile that a publisher site keeps about him. o The enduser requests a service specific to a publisher identity (e.g., nfl.com), but the publisher prohibits the service (e.g., through a "NO OPES" application header). As in the case above, Barbir, et al. Expires August 11, 2003 [Page = 14] =0C Internet-Draft Policy Requirements February = 2003 the publisher must be able to request and receive profile information that a user keeps about a publisher. In general, the PDP should keep its policy base in a manner that makes the decision procedure for all cases easy to understand. 4.4 Integrity and Encryption 4.4.1 Integrity and confidentiality of authentication and requests/ responses for service The requests and responses should be cryptographically tied to the identities of the requestor and responder, and the messages should not alterable without detection. A certificate-based digital signature is strongly recommended as part of the authentication process. A binding between the request and response should be established using well-founded cryptographic means, to show that the response is made in reply to a specific request. 4.4.2 Integrity and confidentiality of application content As directed by the PEP, content will be transformed in whole or in part by OPES services. This means that end-to-end cryptographic protections cannot be used. This is probably acceptable for the = vast majority of traffic, but in cases where a lesser form of content protection is desirable, hop-by-hop protections can be used instead. The requirements for such protections are: o Integrity using shared secrets MUST be used between all = processing points, end-to-end (i.e., the two ends a "hop" must share a secret, but the secret can be different between "hops"). The processing points include the callout servers. o Encryption can be requested separately, with the same secret sharing requirement between "hops". When requested, encryption applies to all processing points, including callout servers. o The signal for integrity (and optionally encryption) must originate from either the requestor (in which case it is applied to the response as well) or the responder (in which case it = covers only the response). o The shared secrets must be unique (to within a very large probabilistic certainty) for each requestor/responder pair. This helps to protect the privacy of enduser data from insider attacks or configuration errors while it transits the provider's network. Barbir, et al. Expires August 11, 2003 [Page = 15] =0C Internet-Draft Policy Requirements February = 2003 4.5 Privacy The PDP must have a privacy policy regarding OPES data such as user profiles for services. Users MUST be able to limit the promulgation of their profile data and their identities. Supported limitations MUST include: o Identity MAY not be given to callout servers. o Profile information MAY not be shared. o Traffic data MAY not be sent to callout servers run by third parties. o Traffic from particular sites SHOULD not be given to OPES callout servers. When an OPES service is provided by a third-party, it must have a privacy policy and identify itself to upstream and downstream parties, telling them how to access its privacy policy. A mechanism is needed to specify these preferences and a protocol to distribute that (see section 3.3). Barbir, et al. Expires August 11, 2003 [Page = 16] =0C Internet-Draft Policy Requirements February = 2003 Normative References [1] A. Barbir et. al, "An Architecture for Open Pluggable Edge Services (OPES)", Internet-Draft: http://www.ietf.org/ internet-drafts/draft-ietf-opes-architecture-04.txt, December 2002. [2] Floyd, S. and L. Daigle, "IAB Architectural and Policy Considerations for Open Pluggable Edge Services", RFC 3238, January 2002. Barbir, et al. Expires August 11, 2003 [Page = 17] =0C Internet-Draft Policy Requirements February = 2003 Informative References [3] Westerinen, A., Schnizlein, J., Strassner, J., Scherling, M., Quinn, B., Herzog, S., Huynh, A., Carlson, M., Perry, J. and S. Waldbusser, "Terminology for Policy-Based Management", RFC = 3198, November 2001. [4] Fielding, R., Gettys, J., Mogul, J., Nielsen, H., Masinter, L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. Authors' Addresses Abbie Barbir Nortel Networks 3500 Carling Avenue Nepean, Ontario K2H 8E9 Canada Phone: +1 613 763 5229 EMail: abbieb@nortelnetworks.com Oskar Batuner Consultant EMail: batuner@attbi.com Andre Beck Lucent Technologies 101 Crawfords Corner Road Holmdel, NJ 07733 USA EMail: abeck@bell-labs.com Tat Chan Nokia 5 Wayside Road Burlington, MA 01803 USA EMail: Tat.Chan@nokia.com Barbir, et al. Expires August 11, 2003 [Page = 18] =0C Internet-Draft Policy Requirements February = 2003 Hilarie Orman Purple Streak Development Phone: EMail: ho@alum.mit.edu Barbir, et al. Expires August 11, 2003 [Page = 19] =0C Internet-Draft Policy Requirements February = 2003 Appendix A. Acknowledgements Many thanks to Andreas Terzis, L. Rafalow (IBM), L. Yang (Intel), M. Condry (Intel) and B. Srinivas (Nokia) Barbir, et al. Expires August 11, 2003 [Page = 20] =0C Internet-Draft Policy Requirements February = 2003 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances = of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification = can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. Full Copyright Statement Copyright (C) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph = are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION Barbir, et al. Expires August 11, 2003 [Page = 21] =0C Internet-Draft Policy Requirements February = 2003 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Barbir, et al. Expires August 11, 2003 [Page = 22] =0C ------_=_NextPart_000_01C2D12A.E490426A-- From owner-ietf-openproxy@mail.imc.org Mon Feb 10 13:41:46 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA20163 for ; Mon, 10 Feb 2003 13:41:39 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1AHsgE10359 for ietf-openproxy-bks; Mon, 10 Feb 2003 09:54:42 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1AHsfd10355 for ; Mon, 10 Feb 2003 09:54:41 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1AHs5o11380; Mon, 10 Feb 2003 12:54:05 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <1LM3FAG5>; Mon, 10 Feb 2003 12:54:05 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404D51588@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Internet-Drafts@ietf.org Cc: "'Markus Hofmann'" , "'Marshall Rose'" , "'OPES Group'" , "Abbie Barbir" Subject: publish draft-ietf-opes-threats-02 Date: Mon, 10 Feb 2003 12:54:03 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C2D12D.661D4826" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C2D12D.661D4826 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D12D.661D4826" ------_=_NextPart_001_01C2D12D.661D4826 Content-Type: text/plain hi, please publish the attached OPES WG draft as "draft-ietf-opes-threats-02". thanks Abbie Nortel Networks ------_=_NextPart_001_01C2D12D.661D4826 Content-Type: text/html publish draft-ietf-opes-threats-02

hi,

please publish the attached OPES WG draft as "draft-ietf-opes-threats-02".

thanks
Abbie
Nortel Networks

  ------_=_NextPart_001_01C2D12D.661D4826-- ------_=_NextPart_000_01C2D12D.661D4826 Content-Type: text/plain; name="draft-ietf-opes-threats-02.txt" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="draft-ietf-opes-threats-02.txt" Content-Transfer-Encoding: quoted-printable Network Working Group A. = Barbir Internet-Draft Nortel = Networks Expires: August 11, 2003 O. = Batuner Independent = consultant B. = Srinivas = Nokia M. = Hofmann Bell Labs/Lucent = Technologies H. = Orman Purple Streak = Development February 10, = 2003 Security Threats and Risks for OPES draft-ietf-opes-threats-02 Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that = other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six = months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on August 11, 2003. Copyright Notice Copyright (C) The Internet Society (2003). All Rights Reserved. Abstract The document investigates the security threats associated with OPES. The effects of security threats on the underlying architecture are discussed. The main goal of this document is threat discovery and analysis. The document does not specify or recommend any solutions. Barbir, et al. Expires August 11, 2003 [Page = 1] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . = 3 2. OPES Data Flow Threats . . . . . . . . . . . . . . . . . . = 5 2.1 OPES Flow Network Level Threats . . . . . . . . . . . . . = 6 2.1.1 Connection-Flow Denial-of-Service (DoS) . . . . . . . . . = 6 2.1.2 Threats to network robustness . . . . . . . . . . . . . . = 7 2.2 OPES Flow Application Level Threats . . . . . . . . . . . = 7 2.2.1 Unauthorized OPES entities . . . . . . . . . . . . . . . . = 7 2.2.2 Unauthorized actions of legitimate OPES entities . . . . . = 7 2.2.3 Unwanted content transformations . . . . . . . . . . . . . = 8 2.2.4 Corrupted content . . . . . . . . . . . . . . . . . . . . = 8 2.2.5 Threats to message structure integrity . . . . . . . . . . = 8 2.2.6 Granularity of protection . . . . . . . . . . . . . . . . = 9 2.2.7 Risks of hop-by-hop protection . . . . . . . . . . . . . . = 9 2.2.8 Threats to integrity of complex data . . . . . . . . . . . = 10 2.2.9 Denial of Service (DoS) . . . . . . . . . . . . . . . . . = 10 2.2.10 Tracing and notification information . . . . . . . . . . . = 10 2.2.11 Unauthenticated Communication in OPES Flow . . . . . . . . = 10 3. Threats to out-of-band data . . . . . . . . . . . . . . . = 11 3.1 Threats that endanger the OPES data flow . . . . . . . . . = 11 3.2 Inaccurate Accounting Information . . . . . . . . . . . . = 11 3.3 OPES service request repudiation . . . . . . . . . . . . . = 12 3.4 Inconsistent privacy policy . . . . . . . . . . . . . . . = 12 3.5 Exposure of privacy preferences . . . . . . . . . . . . . = 12 3.6 Exposure of security settings . . . . . . . . . . . . . . = 12 3.7 Improper enforcement of privacy and security policy . . . = 13 3.8 DOS Attacks . . . . . . . . . . . . . . . . . . . . . . . = 13 4. Security Considerations . . . . . . . . . . . . . . . . . = 14 Normative References . . . . . . . . . . . . . . . . . . . = 15 Informative References . . . . . . . . . . . . . . . . . . = 16 Authors' Addresses . . . . . . . . . . . . . . . . . . . . = 16 A. Acknowledgements . . . . . . . . . . . . . . . . . . . . . = 18 Intellectual Property and Copyright Statements . . . . . . = 19 Barbir, et al. Expires August 11, 2003 [Page = 2] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 1. Introduction The Open Pluggable Edge Services (OPES) [1] architecture enables cooperative application services (OPES services) between a data provider, a data consumer, and zero or more OPES processors. The application services under consideration analyze and possibly transform application-level messages exchanged between the data provider and the data consumer. The OPES processor can distribute the responsibility of service execution by communicating and collaborating with one or more remote callout servers. The details = of the OPES architecture can be found in [1]. Security threats with respect to OPES can be viewed from different angles. There are security risks that affect content consumer applications, and those that affect the data provider applications. These threats affect the quality and integrity of data that the applications either produce or consume. On the other hand, the security risks can also be categorized into trust within the system (i.e. OPES service providers) and protection of the system from threats imposed by outsiders such as hackers and attackers. Insiders are those parties that are part of the OPES system. Outsiders are those entities that are not participating in the OPES system. It must be noted that not everyone in an OPES delivery path is equally trusted. Each OPES administrative trust domain must protect itself from all outsiders. Furthermore, it may have limited trust relationship with another OPES administrative domain for certain purposes. OPES service provide must use authentication as the basis for building trust relationships between administrative domains. Insiders can intentionally or unintentionally inflict harm and = damage on the data consumer and data provider applications. This can be through bad system configuration, execution of bad software or, if their networks are compromised, by inside or outside hackers. Depending on the deployment scenario, the trust within the OPES system is based on a set of transitive trust relationships between the data provider application, the OPES entities and the data consumer application. Threats to OPES entities can be at the OPES flow level and/or at the network level. In considering threats to the OPES system, the document will follow = a threat analysis model that identifies the threats from the perspective of how they will affect the data consumer and the data provider applications. The main goal of this document is threat discovery and analysis. = The Barbir, et al. Expires August 11, 2003 [Page = 3] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 document does not specify or recommend any solutions. It is important to mention that the OPES architecture has many similarities with other so called overlay networks, specifically web caches and content delivery networks (CDN) (see [2] , [4] ). This document focuses on threats that are introduced by the existence of the OPES processor and callout servers. Security threats specific to content services that do not use the OPES architecture are = considered out-of-scope of this document. However, this document can be used as input when considering security implications for web caches and = CDNs. The document is organized as follows: Section 2 discusses threats to OPES data flow on network and application level, section 3 discusses threats to other parts of the system and section 4 discusses = security considerations. Barbir, et al. Expires August 11, 2003 [Page = 4] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 2. OPES Data Flow Threats Threats to OPES data flow can affect the data consumer and data provider applications. At the OPES flow level, threats can occur at Policy Enforcement Points and Policy Decision Points [3] and along the OPES flow path where network elements are used to process the data. A serious problem is posed by the very fact that the OPES architecture is based on widely adopted protocols (HTTP is used as = an example). The architecture document specifically requires that "the presence of an OPES processor in the data request/response flow = SHALL NOT interfere with the operations of non-OPES aware clients and servers". This greatly facilitates OPES deployment but on the other hand a vast majority of clients (browsers) will not be able to exploit any safeguards added as base protocol extensions. For the usual data consumer, who might have questions such as (Where does this content come from? Can I get it another way? What is the difference? Is it legitimate?). Even if there are facilities and technical expertise present to pursue these questions, such thorough examination of each result is prohibitively expensive in terms of time and effort. OPES-aware content providers may try to protect themselves by adding verification scripts and special page structures. OPES-aware end users may use special tools. In all other cases (non-OPES aware clients and servers) protection will rely on monitoring services and investigation of occasionally discovered anomalies. An OPES system poses a special danger as a possible base for classical man-in-the-middle attacks. One of the reasons why such attacks are relatively rare is the difficulty in finding an appropriate base: a combination of a traffic interception point controlling a large flow of data and an application codebase running on a high-performance hardware with sufficient performance to analyze and possible modify all passing data. An OPES processor = meets this definition. This calls for special attention to protection measures at all levels of the system. Any compromise of an OPES processor or remote callout server can = have a ripple effect on the integrity of the affected OPES services = across all service providers that use the service. To mitigate this threat appropriate security procedures and tools (e.g., a firewall) should be applied. Specific threats can exist at the network level and at OPES data = flow level. Barbir, et al. Expires August 11, 2003 [Page = 5] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 2.1 OPES Flow Network Level Threats OPES processor and callout servers are susceptible to network level attacks from outsiders or from the networks of other OPES service providers (i.e. if the network of a contracted OPES service is compromised). The OPES architecture is based on common application protocols that do not provide strong guarantees of privacy, authentication, or integrity. The IAB considerations [4] require that the IP address of an OPES processor be accessible to data consumer applications at the IP addressing level. This requirement limit the ability of service providers of positioning the OPES processor behind firewalls and may exposes the OPES processor, including remote callout servers, to network level attacks. For example, the use of TCP/IP as network level protocol makes OPES processors subject to many known attacks, such as IP spoofing and session stealing. The OPES system is also susceptible to a number of security threats that are commonly associated with network infrastructure. These threats include snooping, denial of service, sabotage, vandalism, industrial espionage and theft of service. There are best practice solutions to mitigate network level threats. It is recommended that the security of the OPES entities at the network level be enhanced using known techniques and methods that minimize the risks of IP spoofing, snooping, denial of service and session stealing. At the OPES Flow level, connection-level security between the OPES processor and callout servers is an important consideration. For example, it is possible to spoof the OPES processor or the remote callout server. There are threats to data confidentiality between = the OPES processor and the remote callout server in an OPES flow. The next subsections covers possible DOS attack on OPES processor, remote callout server or data consumer application and network robustness. 2.1.1 Connection-Flow Denial-of-Service (DoS) OPES processors, or callout servers or data consumer applications = can be vulnerable to DOS attacks. DOS attacks can be of various types. One example of DOS attacks is the overloading of OPES processors or callout servers by spurious service requests issued by a malicious node, which denies the legal data traffic the necessary resources to render service. The resources include CPU cycles, memory, network interfaces, etc. A Denial-of-Service attack can be selective, Barbir, et al. Expires August 11, 2003 [Page = 6] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 generic or random in terms of which communication streams are affected. Distributed DoS is also possible when an attacker successfully directs multiple nodes over the network to initiate spurious service requests to an OPES processor(or callout server) simultaneously. 2.1.2 Threats to network robustness If OPES implementation does violate end-to-end addressing = principles, it could endanger the Internet infrastructure by complicating = routing and connection management. If it does not use flow-control principles for managing connections, or if it interferes with end-to-end flow control of connections that it did not originate, then it could cause Internet congestion. An implementation that violates the IAB requirement of explicit IP level addressing (for example by adding OPES functional capabilities to an interception proxy) may defeat some of the protective mechanisms and safeguards built into the OPES architecture. 2.2 OPES Flow Application Level Threats At the content level, threats to the OPES system can come from outsiders or insiders. The threat from outsiders is frequently intentional. Threats from insiders can be intentional or due to inappropriate implementations such as programming and configuration errors that result in bad system behavior. Application level problems and threats to the OPES systems are discussed below: 2.2.1 Unauthorized OPES entities Although one party authorization is mandated by the OPES architecture, such authorization occurs out-of-band. Discovering = the presence of an OPES entity and verifying authorization requires special actions and may present a problem. Adding notification and authorization information to the data messages (by using base protocol extensions) may help, especially if the data consumer's software is aware of such extensions. 2.2.2 Unauthorized actions of legitimate OPES entities According to the OPES architecture, the authorization is not tightly coupled with specific rules and procedures triggered by the rules. Even if a requirement to approve each particular rule and procedure Barbir, et al. Expires August 11, 2003 [Page = 7] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 was set, it looks at least impractical if not impossible to request such permission from the end user. The authorization is basically given for the class of transformations. The actual rules and triggered procedures may (maliciously or due to a programming error) perform actions that they are not authorized for. 2.2.3 Unwanted content transformations An authorized OPES service may perform actions that do not adhere to the expectations of the party that gave the authorization for the service. Examples may include ad flooding by a local ad insertion service or use of inappropriate policy by a content filtering service. On the other hand an OPES entity acting on behalf of one party may perform transformations that another party deems inappropriate. Examples may include replacing ads initially inserted by the content provider or applying filtering transformations that change the meaning of the text. 2.2.4 Corrupted content The OPES system may deliver outdated or otherwise distorted information due to programming problems or as a result of malicious attacks. For example, a compromised server, instead of performing OPES service, may inject bogus content. Such an action may be an act of cyber-vandalism (including virus injection) or intentional distribution of misleading information (such as manipulations with financial data). A compromised OPES server or malicious entity in the data flow may introduce changes specifically intended to cause improper actions in the OPES server or callout server. These changes may be in the message body, headers or both. This type of threat is discussed in more detail below. 2.2.5 Threats to message structure integrity An OPES server may add, remove or delete certain headers in a = request and/or response message (for example to implement additional privacy protection or assist in content filtering). Such changes may violate end-to-end integrity requirements or defeat services that use information provided in such headers (for example some local filtering services or reference-based services). Barbir, et al. Expires August 11, 2003 [Page = 8] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 2.2.6 Granularity of protection OPES services have implicit permission to modify content. However, the permissions generally apply only to portions of the content, for example, URL's between particular HTML tags, or text in headlines, = or URL's matching particular patterns. In order to express such policies, one must be able to refer to portions of messages and to detect modifications to message parts. Because there is currently very little support for policies that are expressed in terms of message parts, it will be difficult to attribute any particular modification to a particular OPES = processor, or to automatically detect policy violations. A fine-grained policy language should be devised, and it could be enforced using digital signatures. This would avoid the problems inherent in hop-by-hop data integrity measures (see next section). 2.2.7 Risks of hop-by-hop protection OPES services cannot be applied to data protected with end-to-end encryption methods because, by definition, the decryption key cannot be shared with OPES processors. This means that if the endpoint policies permit OPES services, the data must either be transmitted without confidentiality protections or else with an alternative to end-to-end encryption: hop-by-hop encryption. In the latter case, = all the parties in the OPES processing path must understand the encryption requirement and negotiate encrypted connections with = their OPES partners. Hop-by-hop protection is less effective than end-to-end protection, because any processor in the path which shares a cryptographic key, can violate the confidentiality or integrity of the data without detection. If a pair of processors in the delivery path use weak cryptography = or manage keys poorly, there is a danger of data leakage. For this reason, different cryptographic keys should be used for each leg of the data stream. Even if the data is not confidential, one might desire some checks = on data integrity, to avoid modifications by unauthorized parties. The comments above apply to the use of end-to-end integrity, if it is based on shared-key cryptography. Again, it should be possible to use hop-by-hop data integrity to protect data as it moves between protection domains. Currently there is no method to signal hop-by-hop encryption or Barbir, et al. Expires August 11, 2003 [Page = 9] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 integrity requirements. Either this must be added to the = application protocol, or OPES must define its own signaling protocol, or all = OPES traffic MUST ALWAYS be encrypted. 2.2.8 Threats to integrity of complex data The OPES system may violate data integrity by applying inconsistent transformations to interrelated data objects or references within = the data object. Problems may range from a broken reference structure (modified/missing targets, references to wrong locations or missing documents) to deliberate replacement/deletion/insertion of links = that violate intentions of the content provider. 2.2.9 Denial of Service (DoS) The data consumer application may not be able to access data if the OPES system fails for any reason. A malicious or malfunctioning node may be able to block all traffic. The data traffic destined for the OPES processor (or callout server) may not be able to use the services of the OPES device. The DoS may be achieved by preventing the data traffic from reaching the processor or the callout server. 2.2.10 Tracing and notification information Inadequate or vulnerable implementation of the tracing and notification mechanisms may defeat safeguards built into the OPES architecture. Tracing and notification facilities may become a target of malicious attack. Such an attack may create problems in discovering and stopping other attacks. The absence of a standard for tracing and notification information may present an additional problem. This information is produced and consumed by the independent entities (OPES servers/user agents/ content provider facilities). This calls for a set of standards related to each base protocol in use. 2.2.11 Unauthenticated Communication in OPES Flow There are risks and threats that could arise from unauthenticated communication between the OPES server and callout servers. Lack of use of strong authentication between OPES processors and callout servers may open security holes whereby DOS and other types of attacks (see sections [2 and 3]) can be performed. Barbir, et al. Expires August 11, 2003 [Page = 10] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 3. Threats to out-of-band data The OPES architecture separates a data flow from a control information flow (loading rulesets, trust establishment, tracing, policy propagation, etc.). There are certain requirements set for the latter but no specific mechanism is prescribed. This gives more flexibility for implementations but creates more burden for implementers and potential customers to ensure that each specific implementation meets all requirements for data security, entity authentication and action authorization. In addition to performing correct actions on the OPES data flow, any OPES implementation has to provide an adequate mechanism to satisfy requirements for out-of-band data and signaling information integrity. Whatever the specific mechanism may be, it inevitably becomes = subject to multiple security threats and possible attacks. The way the threats and attacks may be realized depends on implementation specifics but the resulting harm generally falls into two = categories: threats to OPES data flow and threats to data integrity. The specific threats are: 3.1 Threats that endanger the OPES data flow Any weakness in the implementation of a security, authentication, or authorization mechanism may open the door to the attacks described = in section 2. An OPES system implementation should address all these threats and prove its robustness and ability to withstand malicious attacks or networking and programming problems. 3.2 Inaccurate Accounting Information Collecting and reporting accurate accounting data may be vital when OPES servers are used to extend a business model of content = provider, service provider or as a basis for third party service. Ability to collect and process accounting data is an important part of OPES system functionality. This functionality may be challenged by distortion or destruction of base accounting data (usually logs), processed accounting data, accounting parameters and reporting configuration. As a result a data consumer may be inappropriately charged for viewing content that was not successfully delivered, or a content provider or independent OPES services provider may not be = compensated Barbir, et al. Expires August 11, 2003 [Page = 11] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 for the services performed. OPES system may use accounting information to distribute resources between different consumers or limit resource usage by a specific consumer. In this case an attack on accounting system (by distortion of data or issuing false configuration commands) may result in incorrect resource management and DoS by artificial resource starvation. 3.3 OPES service request repudiation An entity (producer or consumer) makes an authorized request and claim, later, that it did not make that request. As a result an OPES entity may be held liable for unauthorized changes to the data flow, or will be unable to receive compensation for a service. There SHOULD be a clear request that this service is required and there SHOULD be a clear course of action on the behalf of all parties. This action SHOULD have a request, and SHOULD have an action, and SHOULD have a means of repudiation of the request, and SHOULD have a means to specify the effect of the action. 3.4 Inconsistent privacy policy The OPES entities may have privacy policies that are not consistent with the data consumer application or content provider application. Privacy related problems may be further complicated if OPES = entities, content providers and end users belong to different jurisdictions with different requirements and different levels of legal = protection. As a result the end user may not be aware that he or she does not have the expected legal protection. The content provider may be exposed to legal risks due to a failure to comply with regulation which he is not even aware of. 3.5 Exposure of privacy preferences The OPES system may inadvertently or maliciously expose end user privacy settings and requirements. 3.6 Exposure of security settings There are risks that the OPES system may expose end user security settings when handling the request and responses. The user data must be handled as sensitive system information and protected against accidental and deliberate disclosure. Barbir, et al. Expires August 11, 2003 [Page = 12] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 3.7 Improper enforcement of privacy and security policy OPES entities are part of the content distribution system and as = such take on certain obligations to support security and privacy policies mandated by content producer and/or end user. However there is a danger that these policies are not properly implemented and = enforced. The data consumer application may not be aware that its protections are no longer in effect. There is also the possibility of security and privacy leaks due to the accidental misconfiguration or, due to missunderstanding what rules are in effect for a particular user or request. Privacy and security related parts of the systems can be targeted by malicious attacks and ability to withstand such attacks is of paramount importance. 3.8 DOS Attacks DOS attacks can be of various types. One type of DOS attacks can = take effect by overloading the client. For example, an intruder can = direct an OPES processor to issue numerous responses to a client. There is also additional DOS risk from a rule misconfiguration that would = have the OPES processor ignore a data consumer application. Barbir, et al. Expires August 11, 2003 [Page = 13] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 4. Security Considerations This document discusses multiple security and privacy issues related to the OPES services. Barbir, et al. Expires August 11, 2003 [Page = 14] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 Normative References [1] A. Barbir et. al, "An Architecture for Open Pluggable Edge Services (OPES)", Internet-Draft: http://www.ietf.org/ internet-drafts/draft-ietf-opes-architecture-04.txt, Aug 2002. [2] A. Barbir et. al, "OPES Use Cases and Deployment Scenarios", Internet-Draft: http://www.ietf.org/ draft-ietf-opes-scenarios-01, July 2002. [3] A. Barbir et. al, "Requirements for Policy, Authorization and Enforcement of OPES Services", Internet-Draft: http:// www.ietf.org/internet- drafts/ draft-ietf-opes-authorization-00.txt , October 2002. Barbir, et al. Expires August 11, 2003 [Page = 15] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 Informative References [4] Floyd, S. and L. Daigle, "IAB Architectural and Policy Considerations for Open Pluggable Edge Services", RFC 3238, January 2002. Authors' Addresses Abbie Barbir Nortel Networks 3500 Carling Avenue Nepean, Ontario K2H 8E9 Canada Phone: +1 613 763 5229 EMail: abbieb@nortelnetworks.com Oskar Batuner Independent consultant EMail: batuner@attbi.com Bindignavile Srinivas Nokia 5 Wayside Road Burlington, MA 01803 USA EMail: bindignavile.srinivas@nokia.com Markus Hofmann Bell Labs/Lucent Technologies Room 4F-513 101 Crawfords Corner Road Holmdel, NJ 07733 US Phone: +1 732 332 5983 EMail: hofmann@bell-labs.com Barbir, et al. Expires August 11, 2003 [Page = 16] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 Hilarie Orman Purple Streak Development Phone: EMail: ho@alum.mit.edu Barbir, et al. Expires August 11, 2003 [Page = 17] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 Appendix A. Acknowledgements Many thanks to T. Chan (Nokia) and A. Beck (Lucent) Barbir, et al. Expires August 11, 2003 [Page = 18] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances = of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification = can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. Full Copyright Statement Copyright (C) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph = are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION Barbir, et al. Expires August 11, 2003 [Page = 19] =0C Internet-Draft Security Threats and Risks for OPES February = 2003 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC Editor function is currently provided by the Internet Society. Barbir, et al. Expires August 11, 2003 [Page = 20] =0C ------_=_NextPart_000_01C2D12D.661D4826-- From owner-ietf-openproxy@mail.imc.org Tue Feb 11 07:10:12 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA00044 for ; Tue, 11 Feb 2003 07:10:10 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1BBnMI13802 for ietf-openproxy-bks; Tue, 11 Feb 2003 03:49:22 -0800 (PST) Received: from ietf.org (odin.ietf.org [132.151.1.176]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1BBnKd13797 for ; Tue, 11 Feb 2003 03:49:21 -0800 (PST) Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA28568; Tue, 11 Feb 2003 06:45:37 -0500 (EST) Message-Id: <200302111145.GAA28568@ietf.org> Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" To: IETF-Announce: ; Cc: ietf-openproxy@imc.org From: Internet-Drafts@ietf.org Reply-to: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-opes-authorization-02.txt Date: Tue, 11 Feb 2003 06:45:36 -0500 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Open Pluggable Edge Services Working Group of the IETF. Title : Policy, Authorization and Enforcement Requirements of OPES Author(s) : A. Barbir et al. Filename : draft-ietf-opes-authorization-02.txt Pages : 22 Date : 2003-2-10 This document describes policy, authorization and enforcement requirements for the selection of the services to be applied to a given OPES flow. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-opes-authorization-02.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-opes-authorization-02.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-opes-authorization-02.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2003-2-10132921.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-opes-authorization-02.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-opes-authorization-02.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2003-2-10132921.I-D@ietf.org> --OtherAccess-- --NextPart-- From owner-ietf-openproxy@mail.imc.org Tue Feb 11 07:13:06 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA00240 for ; Tue, 11 Feb 2003 07:13:05 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1BBnRX13811 for ietf-openproxy-bks; Tue, 11 Feb 2003 03:49:27 -0800 (PST) Received: from ietf.org (odin.ietf.org [132.151.1.176]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1BBnQd13806 for ; Tue, 11 Feb 2003 03:49:26 -0800 (PST) Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA28587; Tue, 11 Feb 2003 06:45:42 -0500 (EST) Message-Id: <200302111145.GAA28587@ietf.org> Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" To: IETF-Announce: ; Cc: ietf-openproxy@imc.org From: Internet-Drafts@ietf.org Reply-to: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-opes-threats-02.txt Date: Tue, 11 Feb 2003 06:45:42 -0500 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Open Pluggable Edge Services Working Group of the IETF. Title : Security Threats and Risks for Open Author(s) : A. Barbir et al. Filename : draft-ietf-opes-threats-02.txt Pages : 20 Date : 2003-2-10 The document investigates the security threats associated with OPES. The effects of security threats on the underlying architecture are discussed. The main goal of this document is threat discovery and analysis. The document does not specify or recommend any solutions. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-opes-threats-02.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-opes-threats-02.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-opes-threats-02.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2003-2-10132930.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-opes-threats-02.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-opes-threats-02.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2003-2-10132930.I-D@ietf.org> --OtherAccess-- --NextPart-- From owner-ietf-openproxy@mail.imc.org Tue Feb 11 20:41:56 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA24519 for ; Tue, 11 Feb 2003 20:41:55 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1C1JW423273 for ietf-openproxy-bks; Tue, 11 Feb 2003 17:19:32 -0800 (PST) Received: from ietf.org (odin.ietf.org [132.151.1.176]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1C1JVd23269 for ; Tue, 11 Feb 2003 17:19:31 -0800 (PST) Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA23864; Tue, 11 Feb 2003 20:15:48 -0500 (EST) Message-Id: <200302120115.UAA23864@ietf.org> To: IETF-Announce: ; Cc: RFC Editor , Internet Architecture Board , ietf-openproxy@imc.org From: The IESG Subject: Document Action: Requirements for OPES Callout Protocols to Informational Date: Tue, 11 Feb 2003 20:15:47 -0500 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: The IESG has approved the following Internet-Drafts as Informational RFCs: o 'Requirements for OPES Callout Protocols' o 'An Architecture for Open Pluggable Edge Services (OPES)' These documents are the product of the Open Pluggable Edge Services Working Group. The IESG contact persons are Ned Freed and Patrik Faltstrom. An announcement to this effect will be sent shortly. As explained previously, the IESG believes that draft-ietf-opes-scenarios-01.txt is too dependent on draft-ietf-opes-threats-00.txt to publish at this time. The IESG is concerned, however, that what appears in these drafts is a fairly high level overview of OPES requirements and architecture. Missing is any mention of specific mechanisms to provide security and congestions. This is not necessarily a problem given these are informational requirement and architecture documents. However, the IESG notes that the lack of specificity at this point could lead to problems deciding on actual protocol details later in the process. Requirement and architecture documents have often proved to be useful tools in reaching consensus on subsequent protocol specifications. Should difficulties arise in reaching consensus later it may be necessary and useful to expand on these documents with information about specific mechanisms to be used. From owner-ietf-openproxy@mail.imc.org Wed Feb 12 11:03:28 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA07357 for ; Wed, 12 Feb 2003 11:03:25 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1CFUmj20634 for ietf-openproxy-bks; Wed, 12 Feb 2003 07:30:48 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1CFUld20630 for ; Wed, 12 Feb 2003 07:30:47 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1CFUfD29182 for ; Wed, 12 Feb 2003 10:30:42 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <1LM3GBDL>; Wed, 12 Feb 2003 10:30:41 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404DB1C4E@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: ietf-openproxy@imc.org Subject: FW: I-D ACTION:draft-iab-sec-cons-03.txt Date: Wed, 12 Feb 2003 10:30:40 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_000_01C2D2AB.B156C6D0" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C2D2AB.B156C6D0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D2AB.B156C6D0" ------_=_NextPart_001_01C2D2AB.B156C6D0 Content-Type: text/plain fyi, abbie > -----Original Message----- > From: Internet-Drafts@ietf.org [mailto:Internet-Drafts@ietf.org] > Sent: Wednesday, February 12, 2003 9:22 AM > Cc: iab@iab.org > Subject: I-D ACTION:draft-iab-sec-cons-03.txt > > > A New Internet-Draft is available from the on-line > Internet-Drafts directories. This draft is a work item of the > Internet Architecture Board Working Group of the IETF. > > Title : Guidelines for Writing RFC Text on > Security Considerations > Author(s) : E. Rescorla, B. Korver > Filename : draft-iab-sec-cons-03.txt > Pages : 42 > Date : 2003-2-11 > > All RFCs are required to have a Security Considerations > section. His- torically, such sections have been relatively > weak. This document provides guidelines to RFC authors on how > to write a good Security Considerations section. > > A URL for this Internet-Draft is: > http://www.ietf.org/internet-drafts/draft-iab-> sec-cons-03.txt > > > To remove yourself from the IETF > Announcement list, send a message to > ietf-announce-request with the word unsubscribe in the body > of the message. > > Internet-Drafts are also available by anonymous FTP. Login > with the username "anonymous" and a password of your e-mail > address. After logging in, type "cd internet-drafts" and then > "get draft-iab-sec-cons-03.txt". > > A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-iab-sec-cons-03.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. ------_=_NextPart_001_01C2D2AB.B156C6D0 Content-Type: text/html Content-Transfer-Encoding: quoted-printable FW: I-D ACTION:draft-iab-sec-cons-03.txt

fyi,
abbie

> -----Original Message-----
> From: Internet-Drafts@ietf.org [mailto:Internet-Drafts@ietf.org= ]
> Sent: Wednesday, February 12, 2003 9:22 = AM
> Cc: iab@iab.org
> Subject: I-D = ACTION:draft-iab-sec-cons-03.txt
>
>
> A New Internet-Draft is available from the = on-line
> Internet-Drafts directories. This draft is a = work item of the
> Internet Architecture Board Working Group of = the IETF.
>
>       = Title           : = Guidelines for Writing RFC Text on
> Security Considerations
>       = Author(s)       : E. Rescorla, B. = Korver
>       = Filename        : = draft-iab-sec-cons-03.txt
>       = Pages           : = 42
>       = Date            : = 2003-2-11
>      
> All RFCs are required to have a Security = Considerations
> section. His- torically, such sections have = been relatively
> weak. This document provides guidelines to RFC = authors on how
> to write a good Security Considerations = section.
>
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-iab->= sec-cons-03.txt
>
>
> To remove yourself from the IETF
> Announcement list, send a message to
> ietf-announce-request with the word unsubscribe = in the body
> of the message.
>
> Internet-Drafts are also available by anonymous = FTP. Login
> with the username "anonymous" and a = password of your e-mail
> address. After logging in, type "cd = internet-drafts" and then
>       "get = draft-iab-sec-cons-03.txt".
>
> A list of Internet-Drafts directories can be = found in
http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt


Internet-Drafts can also be obtained by = e-mail.

Send a message to:
        mailserv@ietf.org.
In the body type:
        "FILE = /internet-drafts/draft-iab-sec-cons-03.txt".
       =20
NOTE:   The mail server at ietf.org can = return the document in
        MIME-encoded form by using the "mpack" = utility.  To use this
        feature, = insert the command "ENCODING mime" before the = "FILE"
        command.  To decode the response(s), you will need = "munpack" or
        a = MIME-compliant mail reader.  Different MIME-compliant mail = readers
        exhibit = different behavior, especially when dealing with
        "multipart" MIME messages (i.e. documents which have = been split
        up into = multiple messages), so check your local documentation on
        how to = manipulate these messages.
        =        =20
        =        =20
Below is the data which will enable a MIME compliant = mail reader implementation to automatically retrieve the ASCII version = of the Internet-Draft.

  ------_=_NextPart_001_01C2D2AB.B156C6D0-- ------_=_NextPart_000_01C2D2AB.B156C6D0 Content-Type: message/rfc822 To: Subject: Date: Wed, 12 Feb 2003 10:14:00 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/mixed; boundary="----_=_NextPart_002_01C2D2AB.B156C6D0" ------_=_NextPart_002_01C2D2AB.B156C6D0 Content-Type: multipart/alternative; boundary="----_=_NextPart_003_01C2D2AB.B156C6D0" ------_=_NextPart_003_01C2D2AB.B156C6D0 Content-Type: text/plain ------_=_NextPart_003_01C2D2AB.B156C6D0 Content-Type: text/html

  ------_=_NextPart_003_01C2D2AB.B156C6D0-- ------_=_NextPart_002_01C2D2AB.B156C6D0 Content-Type: application/octet-stream; name="ATT10883" Content-Disposition: attachment; filename="ATT10883" Content-type: message/external-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2003-2-11141333.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-iab-sec-cons-03.txt ------_=_NextPart_002_01C2D2AB.B156C6D0 Content-Type: message/external-body; site="internet-drafts"; dir="draft-iab-sec-cons-03"; mode="ftp.ietf.org"; access-type="anon-ftp" ------_=_NextPart_002_01C2D2AB.B156C6D0-- ------_=_NextPart_000_01C2D2AB.B156C6D0-- From owner-ietf-openproxy@mail.imc.org Wed Feb 12 19:16:58 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA19479 for ; Wed, 12 Feb 2003 19:16:57 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1CNq1x10901 for ietf-openproxy-bks; Wed, 12 Feb 2003 15:52:01 -0800 (PST) Received: from dirty.research.bell-labs.com (ns1.research.bell-labs.com [204.178.16.6]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1CNpwd10894 for ; Wed, 12 Feb 2003 15:52:00 -0800 (PST) Received: from grubby.research.bell-labs.com (H-135-104-2-9.research.bell-labs.com [135.104.2.9]) by dirty.research.bell-labs.com (8.12.5/8.12.5) with ESMTP id h1CNq0hN015480 for ; Wed, 12 Feb 2003 18:52:00 -0500 (EST) Received: from bronx.dnrc.bell-labs.com (bronx.dnrc.bell-labs.com [135.180.160.8]) by grubby.research.bell-labs.com (8.11.6/8.11.6) with ESMTP id h1CNpoY60157 for ; Wed, 12 Feb 2003 18:51:50 -0500 (EST) Received: from mhof.com (biena [135.180.160.86]) by bronx.dnrc.bell-labs.com (8.9.3/8.9.3) with ESMTP id SAA05141 for ; Wed, 12 Feb 2003 18:51:48 -0500 (EST) Message-ID: <3E4ADE35.5010002@mhof.com> Date: Wed, 12 Feb 2003 18:52:21 -0500 From: Markus Hofmann User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.2.1) Gecko/20021130 X-Accept-Language: en-us, en MIME-Version: 1.0 To: OPES Group Subject: Start on OPES protocol work Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit Folks, with the first two WG drafts being approved by the IESG (architecture, protocol requirements), and three others under IESG review (scenarios, threats, policy requirements), it's time to get started on the protocol work!! As a starting point, I'd suggest everyone to familiarize herself/himself with the OPES Protocol Requirements Draft (draft-ietf-opes-protocol-reqs-03). It will provide guidance on fundamental questions we'll have to answer before trying to settle/define a protocol, such as - what type of interactions need to be supported (e.g. pure request/response, or request with multiple replies, or...) - what type of data is exchanged in requests and replies - what are the security requirements - etc. We also need to keep performance expectations in mind, which might impact the decision whether to go binary, text, highly structured (e.g. XML) etc. Based on the answers to these questions, we can then discuss and pick the most suitable protocol. I'm hesitating to list "candidate protocols" since it would *not* be a good idea to talk about specific protocols without having discussed the answers to above questions, but in the past the following options have been mentioned at some point (without a specific order!!): - ICAP - ICAP "next generation" - SOAP/XML - "Something" over BEEP - Binary protocol - more... I would expect that we'll have some lively discussion on the protocol issue here on the mailing list, and that we continue and intensify the discussions at a meeting in San Francisco. Also consider this a call for agenda items for the next IETF meeting, and send Marshall and myself agenda requests related to the protocol issue. So please, let's get started here on the mailing list, keep your thoughts and opinions coming. Thanks, Markus From owner-ietf-openproxy@mail.imc.org Fri Feb 14 13:34:37 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00100 for ; Fri, 14 Feb 2003 13:34:35 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1EI24b13242 for ietf-openproxy-bks; Fri, 14 Feb 2003 10:02:04 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1EI23d13236 for ; Fri, 14 Feb 2003 10:02:03 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1EI1vD13442; Fri, 14 Feb 2003 13:01:57 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6SZ3X>; Fri, 14 Feb 2003 13:01:58 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E0BCCC@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Markus Hofmann , OPES Group Subject: RE: Start on OPES protocol work Date: Fri, 14 Feb 2003 13:01:52 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D453.27631376" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D453.27631376 Content-Type: text/plain Given the latest trends in web services and in particular the work that has been done in XML encryption/signature, I would like to suggest to start looking seriously into adopting SOAP/XML (or OPES compliant SOAP envelop/Schema) for the callout protocol. Using SOAP, we can get the flexibility of defining a protocol that is extensible and flexible. Using SOAP security extension can solve the security requirements for OPES. Having the ability of doing incremental signature is an important feature. Tracing and notification can be provided as a service, that can take advantage of the incremental siging ability of XML. I know off hand that performance issues will be used aginst adopting SOAP. However, keep in mind that performance is only one factor and it can be argued that with continous improvments in CPU speed and Hardware acceleration, this will not (or should not) be a major issue. There is a lot of work that is being done now on SOAP security that can be very usefull to OPES. For OPES to be usefull, I think it should be able to interoperate with web services. At the end of the day, OPES provide a service to the data provider/consumer application. I know that SOAP has been looked at before by OPES. As a start, do we have a pointer on the previous work, if yes can someone please advice where we can get it. Thanks Abbie > -----Original Message----- > From: Markus Hofmann [mailto:markus@mhof.com] > Sent: Wednesday, February 12, 2003 6:52 PM > To: OPES Group > Subject: Start on OPES protocol work > > > > Folks, > > with the first two WG drafts being approved by the IESG > (architecture, > protocol requirements), and three others under IESG review > (scenarios, > threats, policy requirements), it's time to get started on the > protocol work!! > > As a starting point, I'd suggest everyone to familiarize > herself/himself with the OPES Protocol Requirements Draft > (draft-ietf-opes-protocol-reqs-03). It will provide guidance on > fundamental questions we'll have to answer before trying to > settle/define a protocol, such as > > - what type of interactions need to be supported (e.g. pure > request/response, or request with multiple replies, or...) > - what type of data is exchanged in requests and replies > - what are the security requirements > - etc. > > We also need to keep performance expectations in mind, which might > impact the decision whether to go binary, text, highly structured > (e.g. XML) etc. > > Based on the answers to these questions, we can then discuss and pick > the most suitable protocol. I'm hesitating to list "candidate > protocols" since it would *not* be a good idea to talk about specific > protocols without having discussed the answers to above > questions, but > in the past the following options have been mentioned at some point > (without a specific order!!): > > - ICAP > - ICAP "next generation" > - SOAP/XML > - "Something" over BEEP > - Binary protocol > - more... > > I would expect that we'll have some lively discussion on the protocol > issue here on the mailing list, and that we continue and > intensify the > discussions at a meeting in San Francisco. > > Also consider this a call for agenda items for the next IETF meeting, > and send Marshall and myself agenda requests related to the protocol > issue. > > So please, let's get started here on the mailing list, keep your > thoughts and opinions coming. > > Thanks, > Markus > > ------_=_NextPart_001_01C2D453.27631376 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: Start on OPES protocol work

Given the latest trends in web services and in = particular the work that has been done in
XML encryption/signature, I would like to suggest to = start looking seriously into adopting SOAP/XML (or OPES compliant SOAP = envelop/Schema) for the callout protocol.


Using SOAP, we can get the flexibility of defining a = protocol that is extensible and flexible. Using SOAP security extension = can solve the security requirements for OPES.

Having the ability of doing incremental signature is = an important feature. Tracing and notification can be provided as a = service, that can take advantage of the incremental siging ability of = XML.

I know off hand that performance issues will be used = aginst adopting SOAP. However, keep in mind that performance is only = one factor and it can be argued that with continous improvments in CPU = speed and Hardware acceleration, this will not (or should not) be a = major issue.

There is a lot of work that is being done now on SOAP = security that can be very usefull to OPES.

For OPES to be usefull, I think it should be able to = interoperate with web services. At the end of the day, OPES provide a = service to the data provider/consumer application.

I know that SOAP has been looked at before by OPES. = As a start, do we have a pointer on the previous work, if yes can = someone please advice where we can get it.


Thanks
Abbie



> -----Original Message-----
> From: Markus Hofmann [mailto:markus@mhof.com]
> Sent: Wednesday, February 12, 2003 6:52 = PM
> To: OPES Group
> Subject: Start on OPES protocol work
>
>
>
> Folks,
>
> with the first two WG drafts being approved by = the IESG
> (architecture,
> protocol requirements), and three others under = IESG review
> (scenarios,
> threats, policy requirements), it's time to get = started on  the
> protocol work!!
>
> As a starting point, I'd suggest everyone to = familiarize
> herself/himself with the OPES Protocol = Requirements Draft
> (draft-ietf-opes-protocol-reqs-03). It will = provide guidance on
> fundamental questions we'll have to answer = before trying to
> settle/define a protocol, such as
>
>    - what type of interactions = need to be supported (e.g. pure
>      request/response, = or request with multiple replies, or...)
>    - what type of data is = exchanged in requests and replies
>    - what are the security = requirements
>    - etc.
>
> We also need to keep performance expectations = in mind, which might
> impact the decision whether to go binary, text, = highly structured
> (e.g. XML) etc.
>
> Based on the answers to these questions, we can = then discuss and pick
> the most suitable protocol. I'm hesitating to = list "candidate
> protocols" since it would *not* be a good = idea to talk about specific
> protocols without having discussed the answers = to above
> questions, but
> in the past the following options have been = mentioned at some point
> (without a specific order!!):
>
>   - ICAP
>   - ICAP "next = generation"
>   - SOAP/XML
>   - "Something" over = BEEP
>   - Binary protocol
>   - more...
>
> I would expect that we'll have some lively = discussion on the protocol
> issue here on the mailing list, and that we = continue and
> intensify the
> discussions at a meeting in San = Francisco.
>
> Also consider this a call for agenda items for = the next IETF meeting,
> and send Marshall and myself agenda requests = related to the protocol
> issue.
>
> So please, let's get started here on the = mailing list, keep your
> thoughts and opinions coming.
>
> Thanks,
>    Markus
>
>

------_=_NextPart_001_01C2D453.27631376-- From owner-ietf-openproxy@mail.imc.org Fri Feb 14 15:39:46 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04342 for ; Fri, 14 Feb 2003 15:39:45 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1EKC3i19251 for ietf-openproxy-bks; Fri, 14 Feb 2003 12:12:03 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1EKC2d19244 for ; Fri, 14 Feb 2003 12:12:02 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1EKh4nT007173; Fri, 14 Feb 2003 13:43:05 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1EKCjj15020; Fri, 14 Feb 2003 13:12:45 -0700 Date: Fri, 14 Feb 2003 13:12:45 -0700 Message-Id: <200302142012.h1EKCjj15020@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: abbieb@nortelnetworks.com Cc: ietf-openproxy@imc.org In-reply-to: Yourmessage <87609AFB433BD5118D5E0002A52CD75404E0BCCC@zcard0k6.ca.nortel.com> Subject: RE: Start on OPES protocol work Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: I mention that, on Fri, 14 Feb 2003 at 13:01:52 -0500, Abbie Barbir said: > Using SOAP, we can get the flexibility of defining a protocol that is > extensible and flexible. Using SOAP security extension can solve the > security requirements for OPES. > Having the ability of doing incremental signature is an important feature. > Tracing and notification can be provided as a service, that can take > advantage of the incremental siging ability of XML. I've got a book published late last year on XML security, and it does not mention incremental signatures. What are they, how would they be useful to OPES? How can we keep track of this W3C work-in-progress from the IETF vantage point? I'm not a member of W3C, and I cannot judge the maturity of their drafts, the likelihood of standardization of any of the documents. Can you and others promise to serve as liasons, keeping us apprised of what's going on? Or perhaps there's already someone who does this for the XML security work, like Don Eastlake. > I know off hand that performance issues will be used aginst adopting SOAP. > However, keep in mind that performance is only one factor and it can be > argued that with continous improvments in CPU speed and Hardware > acceleration, this will not (or should not) be a major issue. I'm not objecting to examining performance vs. ease-of-use, but it is important to keep the whole picture in mind. If a single CPU with no special hardware enhancements can keep a 2 Gbps line busy, carefully balancing CPU cycles and I/O, then you have to be very careful about adding extra per packet processing that multiplies the CPU cycles by a factor of 1000 - this would mean that an organization would need many more CPU's to do the same job. So let's be very specific when we discuss performance and acceleration - the numbers are important because they determine the cost of the deployed systems. > There is a lot of work that is being done now on SOAP security that can be > very usefull to OPES. How can we know what that very useful work is? > For OPES to be usefull, I think it should be able to interoperate with web > services. At the end of the day, OPES provide a service to the data > provider/consumer application. Can you tell us what it means to interoperate with web services? I'm not against it, but I've got only the vaguest notion of what it might mean for OPES. > I know that SOAP has been looked at before by OPES. As a start, do we have a > pointer on the previous work, if yes can someone please advice where we can > get it. My memory of this is that Lee Rafalow was a strong advocate of looking at SOAP, but there was no work beyond that advocacy. Perhaps others have done work since then, but have not reported it to OPES because we were concentrating only on the initial documents. We need to spread the word that we are now in a new WG phase and it is safe to come back to the mailing list. Hilarie From owner-ietf-openproxy@mail.imc.org Fri Feb 14 19:15:19 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA10239 for ; Fri, 14 Feb 2003 19:15:18 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1ENpED26222 for ietf-openproxy-bks; Fri, 14 Feb 2003 15:51:14 -0800 (PST) Received: from shego.dbc.mtview.ca.us (adsl-64-168-10-254.dsl.scrm01.pacbell.net [64.168.10.254]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1ENpDd26218 for ; Fri, 14 Feb 2003 15:51:13 -0800 (PST) Received: from dhcp13.dbc.mtview.ca.us (localhost [127.0.0.1]) by shego.dbc.mtview.ca.us (8.12.6/8.12.6) with SMTP id h1ENpE45014489; Fri, 14 Feb 2003 15:51:14 -0800 (PST) Date: Fri, 14 Feb 2003 15:51:14 -0800 From: Marshall Rose To: ietf-openproxy@imc.org Cc: Markus Hofmann Subject: Re: Start on OPES protocol work Message-Id: <20030214155114.5045e3e0.mrose@dbc.mtview.ca.us> In-Reply-To: <3E4ADE35.5010002@mhof.com> References: <3E4ADE35.5010002@mhof.com> Organization: Dover Beach Consulting, Inc. X-Mailer: Sylpheed version 0.8.8claws (GTK+ 1.2.10; i386--netbsdelf) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit > I would expect that we'll have some lively discussion on the protocol > issue here on the mailing list, and that we continue and intensify the > discussions at a meeting in San Francisco. > > Also consider this a call for agenda items for the next IETF meeting, > and send Marshall and myself agenda requests related to the protocol > issue. > > So please, let's get started here on the mailing list, keep your > thoughts and opinions coming. just so everyone's clear on this: the "fun" part now begins. however, markus and i need a reason to schedule a wg meeting in san francisco, and we need that reason by february 25th (11 days from now). what this means is that the metabolism of this list needs to pick-up substantially so we know there will be something to actually discuss in san francisco... /mtr From owner-ietf-openproxy@mail.imc.org Fri Feb 14 20:55:47 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA11952 for ; Fri, 14 Feb 2003 20:55:44 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1F1SLs28626 for ietf-openproxy-bks; Fri, 14 Feb 2003 17:28:21 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1F1SKd28622 for ; Fri, 14 Feb 2003 17:28:21 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1F1SBW20070; Fri, 14 Feb 2003 20:28:11 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6TBJZ>; Fri, 14 Feb 2003 20:28:11 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E0C059@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: "The Purple Streak, Hilarie Orman" Cc: ietf-openproxy@imc.org, "Abbie Barbir" Subject: RE: Start on OPES protocol work Date: Fri, 14 Feb 2003 20:28:10 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D491.80004826" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D491.80004826 Content-Type: text/plain Hilarie, see remarks inside, abbie > -----Original Message----- > From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu] > Sent: Friday, February 14, 2003 3:13 PM > To: Barbir, Abbie [CAR:1A00:EXCH] > Cc: ietf-openproxy@imc.org > Subject: RE: Start on OPES protocol work > > > I mention that, on Fri, 14 Feb 2003 at 13:01:52 -0500, Abbie > Barbir said: > > > Using SOAP, we can get the flexibility of defining a > protocol that is > > extensible and flexible. Using SOAP security extension can > solve the > > security requirements for OPES. > > > Having the ability of doing incremental signature is an important > > feature. Tracing and notification can be provided as a > service, that > > can take advantage of the incremental siging ability of XML. > > I've got a book published late last year on XML security, and > it does not mention incremental signatures. What are they, > how would they be useful to OPES? > -------- abbie what I mean by incremental signature is that different parts of the body can be signed by mutiple parties, plus different parts can be exposed to different parties > How can we keep track of this W3C work-in-progress from the > IETF vantage point? I'm not a member of W3C, and I cannot > judge the maturity of their drafts, the likelihood of > standardization of any of the documents. Can you and others > promise to serve as liasons, keeping us apprised of what's > going on? Or perhaps there's already someone who does this > for the XML security work, like Don Eastlake. I acn do what I can, but a lot of the SOAP security work is done in OASIS and they have a open process (I beilive), where u can get access to all the doc. > > > I know off hand that performance issues will be used > aginst adopting > > SOAP. However, keep in mind that performance is only one > factor and > > it can be argued that with continous improvments in CPU speed and > > Hardware acceleration, this will not (or should not) be a major > > issue. > > I'm not objecting to examining performance vs. ease-of-use, > but it is important to keep the whole picture in mind. If a > single CPU with no special hardware enhancements can keep a 2 > Gbps line busy, carefully balancing CPU cycles and I/O, then > you have to be very careful about adding extra per packet > processing that multiplies the CPU cycles by a factor of 1000 > - this would mean that an organization would need many more > CPU's to do the same job. So let's be very specific when we > discuss performance and acceleration - the numbers are > important because they determine the cost of the deployed systems. > --- abbie Before u can do that, we need to know what we are comapring SOAP to. I bet you, that whatever protocol will be developed there will be a need to define a structured way of invoking RPC, which will involve some form of XML or metadata. Even if you ICAP, you still need to know how to find services and how to invoke them, which means some way must be developed to do that. Otherwise, you will be binding the OPES architecture to a group of very limitted services, and you will be forcing the service provider to get hooked on one the services of one vendor. > > There is a lot of work that is being done now on SOAP > security that > > can be very usefull to OPES. > > How can we know what that very useful work is? > ---- abbie SOAP security involves header extensions that allow for Authorization/Authentication information to be exchaged (including info for XML encryption/Sig) There are also provision for Policy exchange, Privacy exchange, Trust and routing. In SOAP, you can use routing information to state which SOAP Actor can be involved in the data path etc. This help OPES since, since it can define which callout server can be used and it can also define what policy to use etc.. (including privacy info...) > > For OPES to be usefull, I think it should be able to interoperate > > with web services. At the end of the day, OPES provide a > service to > > the data provider/consumer application. > > Can you tell us what it means to interoperate with web > services? I'm not against it, but I've got only the vaguest > notion of what it might mean for OPES. > ---- ABBIE WHat I mean here, is that the service provider that deploy OPES, should be able to offer services as they become avilable and as long as they make good financial sence. Since off hand you do not know what a killer APP is, the OPES platform should be extendale, meaning, the OPES provider should be able to offer a service at a minimal cost. SO if SOAP ids the callout protocol, and the new service ids a web service, implementiung it in OPES is quick and strighat fwd. Otherwise, u need to customise the service at leat at the OPES end. WOrst case secnario, the actual provider of the service need to TALK OPES. > > I know that SOAP has been looked at before by OPES. As a > start, do we > > have a pointer on the previous work, if yes can someone > please advice > > where we can get it. > > My memory of this is that Lee Rafalow was a strong advocate > of looking at SOAP, but there was no work beyond that > advocacy. Perhaps others have done work since then, but have > not reported it to OPES because we were concentrating only on > the initial documents. We need to spread the word that we > are now in a new WG phase and it is safe to come back to the > mailing list. > > Hilarie I have access to that info, but i thought that there was a talk about comparing SOAP, ICAP etc.. that was presented in one of the BOFs meetings. abbie > ------_=_NextPart_001_01C2D491.80004826 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: Start on OPES protocol work

Hilarie,

see remarks inside,

abbie

> -----Original Message-----
> From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu]
> Sent: Friday, February 14, 2003 3:13 PM
> To: Barbir, Abbie [CAR:1A00:EXCH]
> Cc: ietf-openproxy@imc.org
> Subject: RE: Start on OPES protocol work
>
>
> I mention that, on Fri, 14 Feb 2003 at 13:01:52 = -0500, Abbie
> Barbir said:
>
> >  Using SOAP, we can get the = flexibility of defining a
> protocol that is 
> > extensible and flexible. Using SOAP = security extension can
> solve the 
> > security requirements for OPES.
>
> >  Having the ability of doing = incremental signature is an important
> > feature.  Tracing and notification = can be provided as a
> service, that
> > can take  advantage of the = incremental siging ability of XML.
>
> I've got a book published late last year on XML = security, and
> it does not mention incremental = signatures.  What are they,
> how would they be useful to OPES?
>

-------- abbie
what I mean by incremental signature is that = different parts of the body can be signed by mutiple parties, plus = different parts can be exposed to different parties


> How can we keep track of this W3C = work-in-progress from the
> IETF vantage point?  I'm not a member of = W3C, and I cannot
> judge the maturity of their drafts, the = likelihood of
> standardization of any of the documents.  = Can you and others
> promise to serve as liasons, keeping us = apprised of what's
> going on?  Or perhaps there's already = someone who does this
> for the XML security work, like Don = Eastlake.

I acn do what I can, but a lot of the SOAP security = work is done in OASIS and they have a open process (I beilive), where u = can get access to all the doc.

>
> >  I know off hand that performance = issues will be used
> aginst adopting
> > SOAP.  However, keep in mind that = performance is only one
> factor and
> > it can be  argued that with continous = improvments in CPU speed and
> > Hardware  acceleration, this will not = (or should not) be a major
> > issue.
>
> I'm not objecting to examining performance vs. = ease-of-use,
> but it is important to keep the whole picture = in mind.  If a
> single CPU with no special hardware = enhancements can keep a 2
> Gbps line busy, carefully balancing CPU cycles = and I/O, then
> you have to be very careful about adding extra = per packet
> processing that multiplies the CPU cycles by a = factor of 1000
> - this would mean that an organization would = need many more
> CPU's to do the same job.  So let's be = very specific when we
> discuss performance and acceleration - the = numbers are
> important because they determine the cost of = the deployed systems.
>

--- abbie

Before u can do that, we need to know what we are = comapring SOAP to.
I bet you, that whatever protocol will be developed = there will be a need to define a structured way of invoking RPC, which = will involve some form of XML or metadata.

Even if you ICAP, you still need to know how to find = services and how to invoke them, which means some way must be developed = to do that. Otherwise, you will be binding the OPES architecture to a = group of very limitted services, and you will be forcing the service = provider to get hooked on one the services of one vendor.



> >  There is a lot of work that is being = done now on SOAP
> security that
> > can be  very usefull to OPES.
>
> How can we know what that very useful work = is?
>

---- abbie

SOAP security involves header extensions that allow = for Authorization/Authentication information to be exchaged (including = info for XML encryption/Sig)

There are also provision for Policy exchange, Privacy = exchange, Trust and routing.

In SOAP, you can use routing information to state = which SOAP Actor can be involved in the data path etc. This help OPES = since, since it can define which callout server can be used and it can = also define what policy to use etc.. (including privacy = info...)


> >  For OPES to be usefull, I think it = should be able to interoperate
> > with web  services. At the end of the = day, OPES provide a
> service to
> > the data  provider/consumer = application.
>
> Can you tell us what it means to interoperate = with web
> services?  I'm not against it, but I've = got only the vaguest
> notion of what it might mean for OPES.
>

---- ABBIE
WHat I mean here, is that the service provider that = deploy OPES, should be able to offer services as they become avilable = and as long as they make good financial sence. Since off hand you do = not know what a killer APP is, the OPES platform should be extendale, = meaning, the OPES provider should be able to offer a service at a = minimal cost. SO if SOAP ids the callout protocol, and the new service = ids a web service, implementiung it in OPES is quick and strighat fwd. = Otherwise, u need to customise the service at leat at the OPES = end.

WOrst case secnario, the actual provider of the = service need to TALK OPES.


> >  I know that SOAP has been looked at = before by OPES. As a
> start, do we
> > have a  pointer on the previous work, = if yes can someone
> please advice
> > where we can  get it.
>
> My memory of this is that Lee Rafalow was a = strong advocate
> of looking at SOAP, but there was no work = beyond that
> advocacy.  Perhaps others have done work = since then, but have
> not reported it to OPES because we were = concentrating only on
> the initial documents.  We need to spread = the word that we
> are now in a new WG phase and it is safe to = come back to the
> mailing list.
>
> Hilarie


I have access to that info, but i thought that there = was a talk about comparing SOAP, ICAP etc.. that was presented in one = of the BOFs meetings.


abbie



>

------_=_NextPart_001_01C2D491.80004826-- From owner-ietf-openproxy@mail.imc.org Fri Feb 14 22:29:21 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA13645 for ; Fri, 14 Feb 2003 22:29:20 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1F2vVp01276 for ietf-openproxy-bks; Fri, 14 Feb 2003 18:57:31 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1F2vUd01271 for ; Fri, 14 Feb 2003 18:57:30 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1F3SinT018931; Fri, 14 Feb 2003 20:28:45 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1F2wVY01578; Fri, 14 Feb 2003 19:58:31 -0700 Date: Fri, 14 Feb 2003 19:58:31 -0700 Message-Id: <200302150258.h1F2wVY01578@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: abbieb@nortelnetworks.com Cc: ietf-openproxy@imc.org In-reply-to: Yourmessage <87609AFB433BD5118D5E0002A52CD75404E0C059@zcard0k6.ca.nortel.com> Subject: RE: Start on OPES protocol work Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: 1. Can you give us a list of pointers to these XML documents? If it is the XML security specs, I believe that is well-defined, at least at some useful version level from 2001. Is there ongoing work, though? Who can tell us? 2. Performance must be considered in detail. If, at the end, the group consensus is to go with XML, it must be an informed decision, with the performance impact clearly understood. I assert that web caches will be the baseline systems, and that is the only logical place to evaluate performance. 3. We need to have all references to SOAP capabilities backed up by references to a spec. Could someone provide pointers to the necessary documents? 4. Are there IPR issues wrt to SOAP? 5. I need more detail about how web services can help with OPES. I understand that there are directories, query/response protocols, semantics for defining entries in the directories, etc. But can someone elaborate to the next level and describe how OPES will use them? 6. It looks like the original website that we used during the BOF stage has been replaced. It would be really nice to have the contents of that site restored to someplace convenient, because there was some excellent information there. Markus, Abbie, have you got the content from the machine Intel used for us? More inline below: Abbie Barbir, in replying to Hilarie Orman's messages: > > How can we keep track of this W3C work-in-progress from the > > IETF vantage point? I'm not a member of W3C, and I cannot > > judge the maturity of their drafts, the likelihood of > > standardization of any of the documents. Can you and others > > promise to serve as liasons, keeping us apprised of what's > > going on? Or perhaps there's already someone who does this > > for the XML security work, like Don Eastlake. > I acn do what I can, but a lot of the SOAP security work is done in OASIS > and they have a open process (I beilive), where u can get access to all the > doc. > > > > > I know off hand that performance issues will be used > > aginst adopting > > > SOAP. However, keep in mind that performance is only one > > factor and > > > it can be argued that with continous improvments in CPU speed and > > > Hardware acceleration, this will not (or should not) be a major > > > issue. > > > > I'm not objecting to examining performance vs. ease-of-use, > > but it is important to keep the whole picture in mind. If a > > single CPU with no special hardware enhancements can keep a 2 > > Gbps line busy, carefully balancing CPU cycles and I/O, then > > you have to be very careful about adding extra per packet > > processing that multiplies the CPU cycles by a factor of 1000 > > - this would mean that an organization would need many more > > CPU's to do the same job. So let's be very specific when we > > discuss performance and acceleration - the numbers are > > important because they determine the cost of the deployed systems. > > > --- abbie > Before u can do that, we need to know what we are comapring SOAP to. > I bet you, that whatever protocol will be developed there will be a need to > define a structured way of invoking RPC, which will involve some form of XML > or metadata. Not an RPC, and not XML. Yes, some minimal metadata, but probably not on every packet. First, OPES must be compared to not-OPES. My performance goal is to have OPES impose only a few instructions per packet for packets which do not get OPES enhancements. For those that do get OPES processing, my goal is to be close to line speed out-and-back to the callout servers. Of course, this doubles the I/O load; that being nearly unavoidable, unless there's a second bus, perhaps I'd accept a doubling of the CPU load. > Even if you ICAP, you still need to know how to find services and how to > invoke them, which means some way must be developed to do that. Otherwise, > you will be binding the OPES architecture to a group of very limitted > services, and you will be forcing the service provider to get hooked on one > the services of one vendor. > > > There is a lot of work that is being done now on SOAP > > security that > > > can be very usefull to OPES. > > > > How can we know what that very useful work is? > > > ---- abbie > SOAP security involves header extensions that allow for > Authorization/Authentication information to be exchaged (including info for ppp> XML encryption/Sig) > There are also provision for Policy exchange, Privacy exchange, Trust and > routing. > In SOAP, you can use routing information to state which SOAP Actor can be > involved in the data path etc. This help OPES since, since it can define > which callout server can be used and it can also define what policy to use > etc.. (including privacy info...) We have to have the spec and an expert in implementation in order order to appreciate this. > > > For OPES to be usefull, I think it should be able to interoperate > > > with web services. At the end of the day, OPES provide a > > service to > > > the data provider/consumer application. > > > > Can you tell us what it means to interoperate with web > > services? I'm not against it, but I've got only the vaguest > > notion of what it might mean for OPES. > > > ---- ABBIE > WHat I mean here, is that the service provider that deploy OPES, should be > able to offer services as they become avilable and as long as they make good > financial sence. Since off hand you do not know what a killer APP is, the > OPES platform should be extendale, meaning, the OPES provider should be able > to offer a service at a minimal cost. SO if SOAP ids the callout protocol, > and the new service ids a web service, implementiung it in OPES is quick and > strighat fwd. Otherwise, u need to customise the service at leat at the OPES > end. > WOrst case secnario, the actual provider of the service need to TALK OPES. What does that mean? > > > I know that SOAP has been looked at before by OPES. As a > > start, do we > > > have a pointer on the previous work, if yes can someone > > please advice > > > where we can get it. > > > > My memory of this is that Lee Rafalow was a strong advocate > > of looking at SOAP, but there was no work beyond that > > advocacy. Perhaps others have done work since then, but have > > not reported it to OPES because we were concentrating only on > > the initial documents. We need to spread the word that we > > are now in a new WG phase and it is safe to come back to the > > mailing list. > > > > Hilarie > I have access to that info, but i thought that there was a talk about > comparing SOAP, ICAP etc.. that was presented in one of the BOFs meetings. Don't remember it, but the original website has been consumed. > abbie > > From owner-ietf-openproxy@mail.imc.org Sat Feb 15 10:11:00 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA03846 for ; Sat, 15 Feb 2003 10:10:59 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1FEjBv21740 for ietf-openproxy-bks; Sat, 15 Feb 2003 06:45:11 -0800 (PST) Received: from hosting.altserver.com ([209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1FEjAd21733 for ; Sat, 15 Feb 2003 06:45:10 -0800 (PST) Received: from f11v-6-224.d1.club-internet.fr ([213.44.165.224] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18k3Yv-00014O-00 for ietf-openproxy@imc.org; Sat, 15 Feb 2003 06:45:06 -0800 Message-Id: <5.2.0.9.0.20030215132341.029f32c0@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Sat, 15 Feb 2003 15:51:54 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: RE: Start on OPES protocol work In-Reply-To: <87609AFB433BD5118D5E0002A52CD75404E0BCCC@zcard0k6.ca.norte l.com> Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-5D487B53; boundary="=======54541AF5=======" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=======54541AF5======= Content-Type: multipart/alternative; x-avg-checked=avg-ok-5D487B53; boundary="=====================_40854262==.ALT" --=====================_40854262==.ALT Content-Type: text/plain; x-avg-checked=avg-ok-5D487B53; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit I am quite disturbed by the lack of modelization of all this. I do not really understand what an OPES means to be as I understand it starts as a very general process (modification of the data flow) but limited to a peripheral architecture (edge) and to a very limited set of protocols. My personal reading of this is that OPES belong to what I call for years "open network extended services" (ONES) that I specified, operated and/or sold some for nearly 20 years over various environments (pubic net, X.25, Minitel, fax..). The notion of callout protocol is exciting because it permits to standardize things I ran on a per application basis and to interoperate with third party services. But I am afraid that lack of modelization (where does all that fit in the OSI or an extended OSI model) leads to some confusion between different layers. I suppose everyone will accept I suppose there are at least two global layers involved; - a layer concerned by the object transportation aspects (I suggest Hardware + layers 1-7 OSI) - a layer concerned by the interapplication levels and above (operators, decisions, etc) (I suggest above OSI) To respond to your suggestion I would suggest: 1. we take advantage from the different propositions (XML, SOAP, ...) to understand what the web service (and others) layer provides making a difference between what is common to all of them ... 2. and we take advantage from the features each of them to decide if - it is something we would expert from the other too (so it should be added to the socket) - of if this is something something belonging to the callout protocol level. 3. from then on we can list all what they contributed to what is specific to a callout protocol and what is missing both in a strict OPES environment - and in the more global ONES environment (what should help us not to miss possible extensions). This should result in a fairly good description of - which "transport" protocols are possible to use - how to plug them in - what some protocol should support to become OPES compatible. - the specification for any compact or more specific inter-ONES transportation protocol - the description of the callout specific functions. Let take the two maximum cases Hilarie and you consider: - two CPU interacting together under QNX in the same rack - two web services by different operators on different continents through the internet SOAP may not be the best solution in the first case but mandatory in the second one. Yet the metadata to be exchanged to perform the service will be the same (except that dialog/signaling will demand probably a synchronization adapted to each). Telematic synchronization/decoupling of the applications will probably be a very important issue in all this. To come back on performance related points, I suppose it may be more complex than just hardware considerations - two examples: 1. we will need to "receive" information that have not been sent. Very simple example: every page of a site has a gif logo which may have to be changed in jpeg. The protocol should be able to say "I do not transmit a data you will read where I told you to store it" and to handle the error case and verifications.. 2. we should be able to zap data. Example: a security control center becomes overloaded by the data pouring in from a blown-up plant. An OPES extracts and format data for the remote controller of a door. Delayed infos tell that the door flaps. The system reacts adding to the traffic, while there may not be any door anymore. All that junk information should have been zapped. jfc --=====================_40854262==.ALT Content-Type: text/html; x-avg-checked=avg-ok-5D487B53; charset=us-ascii Content-Transfer-Encoding: 8bit I am quite disturbed by the lack of modelization of all this. I do not really understand what an OPES means to be as I understand it starts as a very general process (modification of the data flow) but limited to a peripheral architecture (edge) and to a very limited set of protocols.

My personal reading of this is that OPES belong to what I call for years "open network extended services" (ONES) that I specified, operated and/or sold some for nearly 20 years over various environments (pubic net, X.25, Minitel, fax..).

The notion of callout protocol is exciting because it permits to standardize things I ran on a per application basis and to interoperate with third party services.

But I am afraid that lack of modelization (where does all that fit in the OSI or an extended OSI model) leads to some confusion between different layers.

I suppose everyone will accept I suppose there are at least two global layers involved;

- a layer concerned by the object transportation aspects (I suggest Hardware + layers 1-7 OSI)
- a layer concerned by the interapplication levels and above (operators, decisions, etc) (I suggest above OSI)

To respond to your suggestion I would suggest:

1. we take advantage from the different propositions (XML, SOAP, ...) to understand what the web service (and others) layer provides making a difference between what is common to all of them ...

2. and we take advantage from the features each of them to decide if
    - it is something we would expert from the other too (so it should be added to the socket)
    - of if this is something something belonging to the callout protocol level.

3. from then on we can list all what they contributed to what is specific to a callout protocol and what is missing both in a strict OPES environment - and in the more global ONES environment (what should help us not to miss possible extensions).

This should result in a fairly good description of
- which "transport" protocols are possible to use
- how to plug them in
- what some protocol should support to become OPES compatible.
- the specification for any compact or more specific inter-ONES transportation protocol 

- the description of the callout specific functions.

Let take the two maximum cases Hilarie and you consider:
- two CPU interacting together under QNX in the same rack
- two web services by different operators on different continents through the internet

SOAP may not be the best solution in the first case but mandatory in the second one. Yet the metadata to be exchanged to perform the service will be the same (except that dialog/signaling will demand probably a synchronization adapted to each). Telematic synchronization/decoupling of the applications will probably be a very important issue in all this.

To come back on performance related points, I suppose it may be more complex than just hardware considerations - two examples:

1. we will need to "receive" information that have not been sent.  Very simple example: every page of a site has a gif logo which may have to be changed in jpeg. The protocol should be able to say "I do not transmit a data you will read where I told you to store it" and to handle the error case and verifications..

2. we should be able to zap data. Example: a security control center becomes overloaded by the data pouring in from a blown-up plant. An OPES extracts and format data for the remote controller of a door. Delayed infos tell that the door flaps. The system reacts adding to the traffic, while there may not be any door anymore. All that junk information should have been zapped.

jfc
 --=====================_40854262==.ALT-- --=======54541AF5======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-5D487B53 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.454 / Virus Database: 253 - Release Date: 10/02/03 --=======54541AF5=======-- From owner-ietf-openproxy@mail.imc.org Sat Feb 15 18:28:31 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA08605 for ; Sat, 15 Feb 2003 18:28:29 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1FN6Di12997 for ietf-openproxy-bks; Sat, 15 Feb 2003 15:06:13 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1FN6Ad12993 for ; Sat, 15 Feb 2003 15:06:11 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id XMUVYYZE; 16 Feb 2003 00:06:07 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: AW: Start on OPES protocol work X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Sun, 16 Feb 2003 00:03:01 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C0FE@hermes.webwasher.com> Thread-Topic: Start on OPES protocol work Thread-Index: AcLS8vMYD/GWyJK5Ro+N+u1K9e06sgCUfzqg From: "Martin Stecher" To: "Markus Hofmann" , "OPES Group" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1FN6Cd12994 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit Hi everybody, I like to follow Markus' suggestion to talk about fundamental questions first and not to compare mentioned protocols in the very beginning or list their pros and cons here. So here are just some thoughts about six topics: Markus' first question > what type of interactions need to be supported (e.g. pure > request/response, or request with multiple replies, or...) together with a sentence from section 3.3 of the requirements draft > a request/response MAY consist of one or more messages 1. I believe that a key feature of the protocol will be a "preview" feature or better a "multiple preview/multiple decision point" feature. While this is one request/response pair on an application message we can look at it as several protocol messages and I think it is best to define this feature that way: The Client sends first part of the application message via one protocol message and receives one protocol message response telling to send the rest of the data too. A second protocol message then encapsulates the rest of the application message. Defining it this way should guarantee a very strict and clear protocol message syntax that avoids those complicated phrases like "after sending this, the client has to wait until blah blah" and this should then have a positive effect on protocol implementations that should avoid that messages on an open persistent connection run out-of sync. 2. The same question from Markus points me to another feature that got on my wish list last year: Although I know that we want to concentrate on HTTP, section 3.11 encourages us to define a protocol that could be used for any application-layer protocol. And unlike most application protocols that have one source and one destination, SMTP has one sender but many recipients. An OPES protocol that is prepared to encapsulate SMTP messages should be aware that a callout service may have different results for different recipients. Therefore I'd welcome if the OPES protocol is able (or can be easily extended) to send multiple replies on one request. Next question from Markus: > what type of data is exchanged in requests and replies Maybe not a direct response but this came to my mind here: 3. I like to minimize the message overhead (see my performance considerations in point 5). Together with what I said above in the first section, I think that a message header and footer (if a footer is necessary at all) should be as short as possible. Usually, protocols write a protocol version into each request and response - but why? Isn't it enough to do this once when a callout connection is established.? All the messages within a persistent connection will be of the same version. When we try to keep this in mind, every message could be designed to be a minimum around the payload. Messages should also follow a same general syntax to make parsing as simple and fast as possible and to allow to skip easily message types that are optional. Third question: > what are the security requirements I hope that somebody could elaborate on this one. I have to admit that I do not see the full implications of every single sentence and requirement of section 5 of the requirements draft in detail. How strong must the encryption and authentication be? Do we need to support multiple different encryption methods? Do we need the newest technology here? When looking at what HTTPS does for HTTP, would that kind of SSL encryption be sufficient for the OPES protocol? 4. Whatever the finding regarding these security questions will be, I very much want to follow the "MAY" in section 5.2 of the requirements and keep this optional. Knowing that a common deployment scenario will be a private network between OPES processor and callout server, we should not enforce unnecessary security if it costs more than only a very little of resources. About performance 5. I believe that performance is absolut key! I experienced that customers very much compare the hardware costs of different filtering solutions. And we are talking with this protocol about deployments at central gateways handling thousands of requests per second. If one protocol implementation needs 20% more resources than another this could mean that another three servers have to be deployed. Having the best technology does not guarantee the success as we all know. Not a topic so far: Ease-Of-Implementation 6. Another key to success will be that the OPES procotol can be implemented easily. If the protocol uses fancy new technology that the majority of edge device developers and filter developers does not know yet and if it is a bit too complicated, the protocol will not be accepted. I think that it is much better to build on protocol parts from the application-level protocols we want to encapsulate. Developers that will likely deal with the OPES protocol should recognize parts of the protocol and should feel familiar with some concepts. The OPES protocol implementation will often be an add-on or a rewrite of existing implementations; being able to re-use parts of the existing code will be very helpfull. That's why I propose to build on concepts like chunked transfer encoding and to use SSL as encryption method if possible and so on. Therefore I also do not tend to binary or highly structured approaches. Facit (sorry I have to violate my introduction here and start to name some protocols ;-) I think about a very lightweight but still powerfull and flexible protocol, building on components of HTTP/HTTPS. Only knowing a little about SOAP and BEEP, I think that they are too complex to define a direct base for the OPES protocol, although there seem to be very interesting concepts which could be integrated. I even think that the base message structure of the OPES protocol should be easier than ICAP/1.0 (you may remember that I misliked the Encapsulated header stuff from the very beginning) and now think that even more could be much simpler. In order to fulfill all requirements the protocol will get rich anyway, so let's try to keep the base as simple as possible. Regards Martin From owner-ietf-openproxy@mail.imc.org Sat Feb 15 20:07:03 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA09367 for ; Sat, 15 Feb 2003 20:07:02 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1G0lmK14758 for ietf-openproxy-bks; Sat, 15 Feb 2003 16:47:48 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1G0lkd14753 for ; Sat, 15 Feb 2003 16:47:46 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1G1JGnT020772; Sat, 15 Feb 2003 18:19:16 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1G0mNT29665; Sat, 15 Feb 2003 17:48:23 -0700 Date: Sat, 15 Feb 2003 17:48:23 -0700 Message-Id: <200302160048.h1G0mNT29665@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: martin.stecher@webwasher.com Cc: ietf-openproxy@imc.org, markus@mhof.com In-reply-to: Yourmessage <72992B39BBD9294BB636A960E89AE02E50C0FE@hermes.webwasher.com> Subject: Re: AW: Start on OPES protocol work Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Inimitably, on Sun, 16 Feb 2003 at 00:03:01 +0100 Martin Stecher decried: > Markus' first question > > what type of interactions need to be supported (e.g. pure > > request/response, or request with multiple replies, or...) > together with a sentence from section 3.3 of the requirements draft > > a request/response MAY consist of one or more messages > 1. > I believe that a key feature of the protocol will be a "preview" feature > or better a "multiple preview/multiple decision point" feature. While > this is one request/response pair on an application message we can look > at it as several protocol messages and I think it is best to define this > feature that way: The Client sends first part of the application message > via one protocol message and receives one protocol message response > telling to send the rest of the data too. A second protocol message then > encapsulates the rest of the application message. Yes, preview is important. > Defining it this way should guarantee a very strict and clear protocol > message syntax that avoids those complicated phrases like "after sending > this, the client has to wait until blah blah" and this should then have > a positive effect on protocol implementations that should avoid that > messages on an open persistent connection run out-of sync. "out-of sync"? I'm not sure what the problem is. Could you elaborate, because it's important that we all understand what the requirements mean. > 2. > The same question from Markus points me to another feature that got on > my wish list last year: Although I know that we want to concentrate on > HTTP, section 3.11 encourages us to define a protocol that could be used > for any application-layer protocol. And unlike most application > protocols that have one source and one destination, SMTP has one sender > but many recipients. An OPES protocol that is prepared to encapsulate > SMTP messages should be aware that a callout service may have different > results for different recipients. > Therefore I'd welcome if the OPES protocol is able (or can be easily > extended) to send multiple replies on one request. Hmmmm - I'm not sure that there's a clear need for multiple replies, interesting as it is. Is this example for OPES located between the sender's mail agent and the SMTP server? Or for OPES located between an SMTP server and the users, working on a message sent to a local alias that is a list? Somehow the callout server (or the OPES box?) knows the expansion of the alias? It might be useful for the OPES processor to advise the callout server to cache the data for multiple uses, though. In order to be application agnostic the two OPES entities must agree on an ID for the content - perhaps borrowed from the application, but in an OPES callout protocol field. > Next question from Markus: > > what type of data is exchanged in requests and replies > Maybe not a direct response but this came to my mind here: > 3. > I like to minimize the message overhead (see my performance > considerations in point 5). Together with what I said above in the first > section, I think that a message header and footer (if a footer is > necessary at all) should be as short as possible. Usually, protocols > write a protocol version into each request and response - but why? Isn't > it enough to do this once when a callout connection is established.? All > the messages within a persistent connection will be of the same version. > When we try to keep this in mind, every message could be designed to be > a minimum around the payload. > Messages should also follow a same general syntax to make parsing as > simple and fast as possible and to allow to skip easily message types > that are optional. > Third question: > > what are the security requirements > I hope that somebody could elaborate on this one. I have to admit that I > do not see the full implications of every single sentence and > requirement of section 5 of the requirements draft in detail. > How strong must the encryption and authentication be? Do we need to > support multiple different encryption methods? Do we need the newest > technology here? One good method would be enough for encryption. 3DES or AES would be fine. Something strong enough to keep bright high school students from writing letters to heads of large companies and explaining to them that their computer club broke the key for the days encryption using 25 computers :-) > When looking at what HTTPS does for HTTP, would that kind of SSL > encryption be sufficient for the OPES protocol? > 4. > Whatever the finding regarding these security questions will be, I very > much want to follow the "MAY" in section 5.2 of the requirements and > keep this optional. Knowing that a common deployment scenario will be a > private network between OPES processor and callout server, we should not > enforce unnecessary security if it costs more than only a very little of > resources. > About performance > 5. > I believe that performance is absolut key! I experienced that customers > very much compare the hardware costs of different filtering solutions. > And we are talking with this protocol about deployments at central > gateways handling thousands of requests per second. If one protocol > implementation needs 20% more resources than another this could mean > that another three servers have to be deployed. Having the best > technology does not guarantee the success as we all know. > Not a topic so far: Ease-Of-Implementation > 6. > Another key to success will be that the OPES procotol can be implemented > easily. If the protocol uses fancy new technology that the majority of > edge device developers and filter developers does not know yet and if it > is a bit too complicated, the protocol will not be accepted. > I think that it is much better to build on protocol parts from the > application-level protocols we want to encapsulate. Developers that will > likely deal with the OPES protocol should recognize parts of the > protocol and should feel familiar with some concepts. The OPES protocol > implementation will often be an add-on or a rewrite of existing > implementations; being able to re-use parts of the existing code will be > very helpfull. > That's why I propose to build on concepts like chunked transfer encoding > and to use SSL as encryption method if possible and so on. Therefore I > also do not tend to binary or highly structured approaches. > Facit (sorry I have to violate my introduction here and start to name > some protocols ;-) > I think about a very lightweight but still powerfull and flexible > protocol, building on components of HTTP/HTTPS. > Only knowing a little about SOAP and BEEP, I think that they are too > complex to define a direct base for the OPES protocol, although there > seem to be very interesting concepts which could be integrated. SOAP might be considered complicated, but BEEP seems to be an entirely different and simpler kind of thing. Possibly simpler than HTTP. > I even think that the base message structure of the OPES protocol should > be easier than ICAP/1.0 (you may remember that I misliked the > Encapsulated header stuff from the very beginning) and now think that > even more could be much simpler. > In order to fulfill all requirements the protocol will get rich anyway, > so let's try to keep the base as simple as possible. > Regards > Martin From owner-ietf-openproxy@mail.imc.org Sun Feb 16 01:10:07 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA12556 for ; Sun, 16 Feb 2003 01:10:06 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1G5lwV20264 for ietf-openproxy-bks; Sat, 15 Feb 2003 21:47:58 -0800 (PST) Received: from lifelesslap.robertcollins.net (CPE-203-51-8-97.nsw.bigpond.net.au [203.51.8.97]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1G5ltd20259 for ; Sat, 15 Feb 2003 21:47:55 -0800 (PST) Received: from localhost ([127.0.0.1] ident=robertc) by lifelesslap.robertcollins.net with esmtp (Exim 3.36 #1 (Debian)) id 18kHeT-0008Nk-00; Sun, 16 Feb 2003 16:47:45 +1100 Subject: Re: AW: Start on OPES protocol work From: Robert Collins To: Martin Stecher Cc: Markus Hofmann , OPES Group In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C0FE@hermes.webwasher.com> References: <72992B39BBD9294BB636A960E89AE02E50C0FE@hermes.webwasher.com> Content-Type: text/plain Organization: Syncretize Pty Limited Message-Id: <1045374458.26132.84.camel@localhost> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.1 Date: 16 Feb 2003 16:47:38 +1100 Content-Transfer-Encoding: 7bit Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit On Sun, 2003-02-16 at 10:03, Martin Stecher wrote: > 3. > I like to minimize the message overhead (see my performance > considerations in point 5). Together with what I said above in the first > section, I think that a message header and footer (if a footer is > necessary at all) should be as short as possible. Usually, protocols > write a protocol version into each request and response - but why? IMO, because it reduces the state to be held by the endpoints of the connection. > Isn't > it enough to do this once when a callout connection is established.? All > the messages within a persistent connection will be of the same version. > When we try to keep this in mind, every message could be designed to be > a minimum around the payload. > > Messages should also follow a same general syntax to make parsing as > simple and fast as possible and to allow to skip easily message types > that are optional. Agreed. IMO this has a greater impact on performance than a few bytes for version identification. The thing to remember is that any general purpose protocol will be slower than a specially tuned protocol. We're building a general purpose tool here - so reliability, ease of implementation and interoperability are (IMO) greater priorities. Rob From owner-ietf-openproxy@mail.imc.org Sun Feb 16 07:43:27 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA25679 for ; Sun, 16 Feb 2003 07:43:26 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1GCIVR02447 for ietf-openproxy-bks; Sun, 16 Feb 2003 04:18:31 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1GCITd02443 for ; Sun, 16 Feb 2003 04:18:30 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id U3A5VM4B; 16 Feb 2003 13:18:26 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: Re: Start on OPES protocol work X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Sun, 16 Feb 2003 13:15:23 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C0FF@hermes.webwasher.com> Thread-Topic: AW: Start on OPES protocol work Thread-Index: AcLVfofIUqM/YL0qSY6UtSoDfVJpsAAM0qnA From: "Martin Stecher" To: "OPES Group" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1GCIUd02444 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit > > > 3. > > I like to minimize the message overhead (see my performance > > considerations in point 5). Together with what I said above > in the first > > section, I think that a message header and footer (if a footer is > > necessary at all) should be as short as possible. Usually, protocols > > write a protocol version into each request and response - but why? > > IMO, because it reduces the state to be held by the endpoints of the > connection. I guess that some parameters that are exchanged during connection establishment need to be kept anyway; I think that another version field in the state info will not hurt at all. On the other hand you are certainly right that the version number overhead is not a big issue anyway. I just wanted to give an example. There is usually much more information which is repeated with every request on the same connection. > > > Isn't > > it enough to do this once when a callout connection is > established.? All > > the messages within a persistent connection will be of the > same version. > > When we try to keep this in mind, every message could be > designed to be > > a minimum around the payload. > > > > Messages should also follow a same general syntax to make parsing as > > simple and fast as possible and to allow to skip easily > message types > > that are optional. > > Agreed. IMO this has a greater impact on performance than a few bytes > for version identification. Correct. But let me again argue why also the little additions can hurt: If the OPES client needs to wrap parts of the application message in a structure with parts in the beginning and at the end, this usually means to copy the message data on application layer again to setup the structure. While the OPES client will need to forward huge amount of data to the callout service, reducing the number of message data copies should be helpful. > > The thing to remember is that any general purpose protocol will be > slower than a specially tuned protocol. We're building a > general purpose > tool here - so reliability, ease of implementation and > interoperability > are (IMO) greater priorities. > Yes, they are but this does not mean that the protocol has to be slow. If we also look at these things, I guess that we can have only a little distance to specially tuned protocols. If we fail here the OPES protocol could stay a nice academic experience without relevance to the industry. Martin From owner-ietf-openproxy@mail.imc.org Sun Feb 16 08:21:45 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA25976 for ; Sun, 16 Feb 2003 08:21:43 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1GD0GX02870 for ietf-openproxy-bks; Sun, 16 Feb 2003 05:00:16 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1GD0Ed02866 for ; Sun, 16 Feb 2003 05:00:15 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id ANFPZ9K8; 16 Feb 2003 14:00:13 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: Re: Start on OPES protocol work X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Sun, 16 Feb 2003 13:57:10 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C100@hermes.webwasher.com> Thread-Topic: AW: Start on OPES protocol work Thread-Index: AcLVVKMCNZEWTOfERWuTX+Eiv7lbYAAYOIVQ From: "Martin Stecher" To: "OPES Group" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1GD0Fd02867 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit > > > Defining it this way should guarantee a very strict and > clear protocol > > message syntax that avoids those complicated phrases like > "after sending > > this, the client has to wait until blah blah" and this > should then have > > a positive effect on protocol implementations that should > avoid that > > messages on an open persistent connection run out-of sync. > > "out-of sync"? I'm not sure what the problem is. Could you > elaborate, > because it's important that we all understand what the > requirements mean. With "out-of-sync" I mean that either no more request can be sent on an open connection because client and server are waiting endless time for an event that the other peer will not trigger (again) or a connection shutdown because one peer detected a syntax error by peeking into a wrong part of the data. I know from ICAP that these things could happen. Two examples: Due to the preview feature and the 204 response an ICAP server wants to stop a message transaction early; it sends the 204 response and is prepared for the next request that comes on this connection. Unfortunately it was too lazy in being patient to wait and acknowledge the rest of the data from the first message that it does not longer care about but the ICAP client is still forwarding. The ICAP server tries to interpret that later part of the message data as the next ICAP request header and fails: Server is not longer in sync. Or another example from ICAP: Due to the little complicated "ieos extension" indicating that a message is not larger than the preview which was sent, an ICAP server could be buggy in a way that waits for more data which is not coming. ICAP client and server wait endless time for each other. I saw an implementation where this happened only for the case in which HTTP body size was exactly the size of the preview. > > > 2. > > The same question from Markus points me to another feature > that got on > > my wish list last year: Although I know that we want to > concentrate on > > HTTP, section 3.11 encourages us to define a protocol that > could be used > > for any application-layer protocol. And unlike most application > > protocols that have one source and one destination, SMTP > has one sender > > but many recipients. An OPES protocol that is prepared to > encapsulate > > SMTP messages should be aware that a callout service may > have different > > results for different recipients. > > Therefore I'd welcome if the OPES protocol is able (or can > be easily > > extended) to send multiple replies on one request. > > Hmmmm - I'm not sure that there's a clear need for multiple replies, > interesting as it is. Is this example for OPES located between the > sender's mail agent and the SMTP server? Or for OPES located between > an SMTP server and the users, working on a message sent to a local > alias that is a list? Somehow the callout server (or the OPES box?) > knows the expansion of the alias? This example is for an OPES box that works as an intermediate SMTP gateway. It could be the sender's SMTP server itself ot a later gateway in the forwarding chain. If this box receives message for 20 users of a single domain, it is a bad idea to create 20 copies of the message first and sending one-sender-one-recipient-pairs to the callout service when maybe 18 copies are filtered with the same settings but 2 copies have a different filter result. I'd would like a protocol that allows the box to forward the SMTP message as is and the callout server to reply "Here is a modified message for 18 recipients and a second message copy for the other two recipients following right after." [...] > > > I think about a very lightweight but still powerfull and flexible > > protocol, building on components of HTTP/HTTPS. > > Only knowing a little about SOAP and BEEP, I think that > they are too > > complex to define a direct base for the OPES protocol, > although there > > seem to be very interesting concepts which could be integrated. > > SOAP might be considered complicated, but BEEP seems to be an entirely > different and simpler kind of thing. Possibly simpler than HTTP. I would like an even simpler basic message structure. Please see my argumentation in my response to Robert. Martin From owner-ietf-openproxy@mail.imc.org Sun Feb 16 12:56:18 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA28110 for ; Sun, 16 Feb 2003 12:56:17 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1GHaig18888 for ietf-openproxy-bks; Sun, 16 Feb 2003 09:36:44 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1GHagd18884 for ; Sun, 16 Feb 2003 09:36:42 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id RD6D52VB; 16 Feb 2003 18:36:42 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D5E1.8A686310" Subject: Request/response pairs X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Sun, 16 Feb 2003 18:33:38 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E8FD1C6@hermes.webwasher.com> Thread-Topic: Request/response pairs Thread-Index: AcLV4cc7fq/O7IdJQeime+2d4Gv1Lg== From: "Martin Stecher" To: "OPES Group" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This is a multi-part message in MIME format. ------_=_NextPart_001_01C2D5E1.8A686310 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi. Yet two other questions regarding OPES request/response pairs: - Should the protocol require that there is always a response to each request? =20 While writing the ICAP extension draft, I was informed about an additional method for logging that has been introduced. This method sends only "requests" (better notifications) to the server (to give a REQMOD service information about the downloaded object from a former REQMOD request for logging purpose). This method does not expect any response. =20 Are these messages w/o response something the OPES protocol should consider or allow or forbid explicitly? =20 - While connections MUST only initiated from client to server, what is about requests on that connection? May it be useful to allow the callout server to send a request back to the client? =20 I am thinking here about the keep-alive messages? It is not only in the interest of the OPES device to check whether the server is still alive. The callout server could also be concerned that no new request is coming along an open connection and wants to verify that the client is not down. =20 What are your thoughts on this? Regards Martin ------_=_NextPart_001_01C2D5E1.8A686310 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Request/response pairs

Hi.

Yet two other questions regarding OPES = request/response pairs:

- Should the protocol require that = there is always a response to each
  request?
 
  While writing the ICAP = extension draft, I was informed about an
  additional method for logging = that has been introduced.
  This method sends only = "requests" (better notifications) to the
  server (to give a REQMOD = service information about the downloaded
  object from a former REQMOD = request for logging purpose).
  This method does not expect any = response.
 
  Are these messages w/o response = something the OPES protocol should
  consider or allow or forbid = explicitly?
 
- While connections MUST only = initiated from client to server, what is
  about requests on that = connection?
  May it be useful to allow the = callout server to send a request back
  to the client?
 
  I am thinking here about the = keep-alive messages?
  It is not only in the interest = of the OPES device to check whether
  the server is still alive. The = callout server could also be concerned
  that no new request is coming = along an open connection and wants to
  verify that the client is not = down.
 
What are your thoughts on this?

Regards
Martin

------_=_NextPart_001_01C2D5E1.8A686310-- From owner-ietf-openproxy@mail.imc.org Sun Feb 16 16:17:15 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA29755 for ; Sun, 16 Feb 2003 16:17:14 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1GKuCN28556 for ietf-openproxy-bks; Sun, 16 Feb 2003 12:56:12 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1GKuBd28551 for ; Sun, 16 Feb 2003 12:56:11 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1GLRtnT031960; Sun, 16 Feb 2003 14:27:55 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1GKuq527904; Sun, 16 Feb 2003 13:56:52 -0700 Date: Sun, 16 Feb 2003 13:56:52 -0700 Message-Id: <200302162056.h1GKuq527904@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: martin.stecher@webwasher.com Cc: ietf-openproxy@imc.org In-reply-to: Yourmessage <72992B39BBD9294BB636A960E89AE02E50C100@hermes.webwasher.com> Subject: Re: Start on OPES protocol work Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: 1. I believe that the protocol requirements stipulate the use of independent "channels", and those were put in exactly for the the case you are discussing for preview. If channels don't solve the problem, we need to know why. 2. For the SMTP case, let's suppose that the site policy requires virus scanning, but some subscribers to the OPES box want potential viruses removed, and some simply want them flagged. Assume a message has been sent to several recipients of a domain with an OPES virus scanning service, and its body is denoted as "A" (identified by its hash, I suppose). If it's the OPES processor that holds the user preferences, then it can handle the process of sending A to a callout server for virus scanning and asking for both removal and flagging, possibly in one request, but at worst in two. Ah, I see, you'd like to eliminate the two requests, and have a request type that asks for multiple services and multiple replies. This is different from asking for multiple services and one reply. A notation for this might be something like: Request(data=D; service=A AND service=B) Reply(data=A(B(D))) Request(data=D; service=A; service=B) Reply(data=A(D); data=B(D)) Have I captured the requirement correctly? 3. I know the arguments for simple formats, but please give something specific about your objection to BEEP. My objection to XML encoding is that it requires many instructions to parse it into tag pairs and parameters, to create a parse tree, and to traverse the tree. When comparing this to direct indexing, or even scans for end-of-header, end-of-line, XML is very slow. But, experience has shown that there are distinct limits to what can be accomplished with fixed parameters positions and bit encodings, and even TCP has had a great deal of trouble staying within its header limitations. Up at the application layer, I think some compromise will be necessary. I'd thought BEEP was actually a good point to begin looking for that compromise point. What is so awful about it? Hilarie From owner-ietf-openproxy@mail.imc.org Sun Feb 16 18:22:06 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA01054 for ; Sun, 16 Feb 2003 18:22:02 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1GMmue01473 for ietf-openproxy-bks; Sun, 16 Feb 2003 14:48:56 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1GMmtd01469 for ; Sun, 16 Feb 2003 14:48:55 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1GMmZn04211; Sun, 16 Feb 2003 17:48:35 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6T4VQ>; Sun, 16 Feb 2003 17:48:35 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E0C102@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: "The Purple Streak, Hilarie Orman" Cc: ietf-openproxy@imc.org Subject: RE: Start on OPES protocol work Date: Sun, 16 Feb 2003 17:48:34 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D60D.89070414" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D60D.89070414 Content-Type: text/plain hi, remarks inline. abbie > -----Original Message----- > From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu] > Sent: Friday, February 14, 2003 9:59 PM > To: Barbir, Abbie [CAR:1A00:EXCH] > Cc: ietf-openproxy@imc.org > Subject: RE: Start on OPES protocol work > > > 1. Can you give us a list of pointers to these XML documents? > If it is the XML security specs, I believe that is > well-defined, at least at some useful version level from > 2001. Is there ongoing work, though? Who can tell us? check the xmldsig (RFC 3275), check also http://www.w3.org/Signature/ > > 2. Performance must be considered in detail. If, at the end, > the group consensus is to go with XML, it must be an informed > decision, with the performance impact clearly understood. I > assert that web caches will be the baseline systems, and that > is the only logical place to evaluate performance. > agreed on checking in detail. however, i have a problem with accepting we caches as a baseline, simply OPES is not a web caching system. > 3. We need to have all references to SOAP capabilities backed > up by references to a spec. Could someone provide pointers > to the necessary documents? > see http://www.w3.org/2000/xp/Group/ > 4. Are there IPR issues wrt to SOAP? > W3C has an Royalty free polic, so there should not be any. > 5. I need more detail about how web services can help with > OPES. I understand that there are directories, > query/response protocols, semantics for defining entries in > the directories, etc. But can someone elaborate to the next > level and describe how OPES will use them? > hilarie, i will elaborate later in the week. > 6. It looks like the original website that we used during the > BOF stage has been replaced. It would be really nice to have > the contents of that site restored to someplace convenient, > because there was some excellent information there. Markus, > Abbie, have you got the content from the machine Intel used for us? > the new site is hosted by Nortel. I am the web master. it should have about all the material from the older site. http://standards.nortelnetworks.com/opes/index.htm > More inline below: > > Abbie Barbir, in replying to Hilarie Orman's messages: > > > > How can we keep track of this W3C work-in-progress from the > > > IETF vantage point? I'm not a member of W3C, and I cannot > > > judge the maturity of their drafts, the likelihood of > > > standardization of any of the documents. Can you and others > > > promise to serve as liasons, keeping us apprised of what's > > > going on? Or perhaps there's already someone who does this > > > for the XML security work, like Don Eastlake. > > > I acn do what I can, but a lot of the SOAP security work > is done in > > OASIS and they have a open process (I beilive), where u can get > > access to all the doc. > > > > > > > > I know off hand that performance issues will be used > > > aginst adopting > > > > SOAP. However, keep in mind that performance is only one > > > factor and > > > > it can be argued that with continous improvments in > CPU speed and > > > > Hardware acceleration, this will not (or should not) > be a major > > > > issue. > > > > > > I'm not objecting to examining performance vs. ease-of-use, > > > but it is important to keep the whole picture in mind. If a > > > single CPU with no special hardware enhancements can keep a 2 > > > Gbps line busy, carefully balancing CPU cycles and I/O, then > > > you have to be very careful about adding extra per packet > > > processing that multiplies the CPU cycles by a factor of 1000 > > > - this would mean that an organization would need many more > > > CPU's to do the same job. So let's be very specific when we > > > discuss performance and acceleration - the numbers are > > > important because they determine the cost of the > deployed systems. > > > > > > --- abbie > > > Before u can do that, we need to know what we are > comapring SOAP to. > > I bet you, that whatever protocol will be developed there will be a > > need to define a structured way of invoking RPC, which > will involve > > some form of XML or metadata. > > Not an RPC, and not XML. Yes, some minimal metadata, but > probably not on every packet. > > First, OPES must be compared to not-OPES. > My performance goal is to have OPES impose only a few > instructions per packet for packets which do not get OPES > enhancements. For those that do get OPES processing, my goal > is to be close to line speed out-and-back to the callout > servers. Of course, this doubles the I/O load; that being > nearly unavoidable, unless there's a second bus, perhaps I'd > accept a doubling of the CPU load. > > > Even if you ICAP, you still need to know how to find > services and how > > to invoke them, which means some way must be developed to do that. > > Otherwise, you will be binding the OPES architecture to a group of > > very limitted services, and you will be forcing the > service provider > > to get hooked on one the services of one vendor. > > > > > > There is a lot of work that is being done now on SOAP > > > security that > > > > can be very usefull to OPES. > > > > > > How can we know what that very useful work is? > > > > > > ---- abbie > > > SOAP security involves header extensions that allow for > > Authorization/Authentication information to be exchaged (including > > info for > ppp> XML encryption/Sig) > > There are also provision for Policy exchange, Privacy > exchange, Trust > > and routing. > > > In SOAP, you can use routing information to state which SOAP Actor > > can be involved in the data path etc. This help OPES > since, since it > > can define which callout server can be used and it can also define > > what policy to use etc.. (including privacy info...) > > We have to have the spec and an expert in implementation in > order order to appreciate this. > > > > > > For OPES to be usefull, I think it should be able to > > interoperate > > > > with web services. At the end of the day, OPES provide a > > > service to > > > > the data provider/consumer application. > > > > > > Can you tell us what it means to interoperate with web > > > services? I'm not against it, but I've got only the vaguest > > > notion of what it might mean for OPES. > > > > > > ---- ABBIE > > WHat I mean here, is that the service provider that deploy OPES, > > should be able to offer services as they become avilable > and as long > > as they make good financial sence. Since off hand you do not know > > what a killer APP is, the OPES platform should be > extendale, meaning, > > the OPES provider should be able to offer a service at a minimal > > cost. SO if SOAP ids the callout protocol, and the new > service ids a > > web service, implementiung it in OPES is quick and strighat fwd. > > Otherwise, u need to customise the service at leat at the OPES end. > > > WOrst case secnario, the actual provider of the service > need to TALK > > OPES. > > What does that mean? > > > > > I know that SOAP has been looked at before by OPES. As a > > > start, do we > > > > have a pointer on the previous work, if yes can someone > > > please advice > > > > where we can get it. > > > > > > My memory of this is that Lee Rafalow was a strong advocate > > > of looking at SOAP, but there was no work beyond that > > > advocacy. Perhaps others have done work since then, but have > > > not reported it to OPES because we were concentrating only on > > > the initial documents. We need to spread the word that we > > > are now in a new WG phase and it is safe to come back to the > > > mailing list. > > > > > > Hilarie > > > > I have access to that info, but i thought that there was a > talk about > > comparing SOAP, ICAP etc.. that was presented in one of > the BOFs meetings. > > Don't remember it, but the original website has been consumed. > > > abbie > > > > > > > > ------_=_NextPart_001_01C2D60D.89070414 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: Start on OPES protocol work

hi,

remarks inline.

abbie


> -----Original Message-----
> From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu]
> Sent: Friday, February 14, 2003 9:59 PM
> To: Barbir, Abbie [CAR:1A00:EXCH]
> Cc: ietf-openproxy@imc.org
> Subject: RE: Start on OPES protocol work
>
>
> 1. Can you give us a list of pointers to these = XML documents?
>  If it is the XML security specs, I = believe that is
> well-defined, at least at some useful version = level from
> 2001.  Is there ongoing work, = though?  Who can tell us?

check the xmldsig (RFC 3275), check also http://www.w3.org/Signature/


>
> 2. Performance must be considered in = detail.  If, at the end,
> the group consensus is to go with XML, it must = be an informed
> decision, with the performance impact clearly = understood.  I
> assert that web caches will be the baseline = systems, and that
> is the only logical place to evaluate = performance.
>

agreed on checking in detail. however, i have a = problem with accepting we caches as a baseline, simply OPES is not a = web caching system.

> 3. We need to have all references to SOAP = capabilities backed
> up by references to a spec.  Could someone = provide pointers
> to the necessary documents?
>

see http://www.w3.org/2000/xp/Group/

> 4. Are there IPR issues wrt to SOAP?
>

W3C has an Royalty free polic, so there should not be = any.

> 5. I need more detail about how web services can = help with
> OPES.  I understand that there are = directories,
> query/response protocols, semantics for = defining entries in
> the directories, etc.  But can someone = elaborate to the next
> level and describe how OPES will use = them?
>

hilarie, i will elaborate later in the week.

> 6. It looks like the original website that we = used during the
> BOF stage has been replaced.  It would be = really nice to have
> the contents of that site restored to someplace = convenient,
> because there was some excellent information = there.  Markus,
> Abbie, have you got the content from the = machine Intel used for us?
>


the new site is hosted by Nortel. I am the web = master. it should have about all the material from the older site. http://standards.nortelnetworks.com/opes/index.htm=


> More inline below:
>
> Abbie Barbir, in replying to Hilarie Orman's = messages:
>
> >  > How can we keep track of this = W3C work-in-progress from the
> >  > IETF vantage point?  I'm = not a member of W3C, and I cannot
> >  > judge the maturity of their = drafts, the likelihood of
> >  > standardization of any of the = documents.  Can you and others
> >  > promise to serve as liasons, = keeping us apprised of what's
> >  > going on?  Or perhaps = there's already someone who does this
> >  > for the XML security work, like = Don Eastlake.
>
> >  I acn do what I can, but a lot of = the SOAP security work
> is done in
> > OASIS  and they have a open process = (I beilive), where u can get
> > access to all the  doc.
>
> >  >
> >  > >  I know off hand that = performance issues will be used
> >  > aginst adopting
> >  > > SOAP.  However, keep = in mind that performance is only one
> >  > factor and
> >  > > it can be  argued = that with continous improvments in
> CPU speed and
> >  > > Hardware  = acceleration, this will not (or should not)
> be a major
> >  > > issue.
> >  >
> >  > I'm not objecting to examining = performance vs. ease-of-use,
> >  > but it is important to keep the = whole picture in mind.  If a
> >  > single CPU with no special = hardware enhancements can keep a 2
> >  > Gbps line busy, carefully = balancing CPU cycles and I/O, then
> >  > you have to be very careful = about adding extra per packet
> >  > processing that multiplies the = CPU cycles by a factor of 1000
> >  > - this would mean that an = organization would need many more
> >  > CPU's to do the same job.  = So let's be very specific when we
> >  > discuss performance and = acceleration - the numbers are
> >  > important because they = determine the cost of the
> deployed systems.
> >  >
>
> >  --- abbie
>
> >  Before u can do that, we need to = know what we are
> comapring SOAP to. 
> > I bet you, that whatever protocol will be = developed there will be a
> > need to  define a structured way of = invoking RPC, which
> will involve
> > some form of XML  or metadata.
>
> Not an RPC, and not XML.  Yes, some = minimal metadata, but
> probably not on every packet.
>
> First, OPES must be compared to = not-OPES.
> My performance goal is to have OPES impose only = a few
> instructions per packet for packets which do = not get OPES
> enhancements.  For those that do get OPES = processing, my goal
> is to be close to line speed out-and-back to = the callout
> servers.  Of course, this doubles the I/O = load; that being
> nearly unavoidable, unless there's a second = bus, perhaps I'd
> accept a doubling of the CPU load.
>
> >  Even if you ICAP, you still need to = know how to find
> services and how
> > to  invoke them, which means some way = must be developed to do that.
> > Otherwise,  you will be binding the = OPES architecture to a group of
> > very limitted  services, and you will = be forcing the
> service provider
> > to get hooked on one  the services of = one vendor.
>
>
> >  > >  There is a lot of = work that is being done now on SOAP
> >  > security that
> >  > > can be  very usefull = to OPES.
> >  >
> >  > How can we know what that very = useful work is?
> >  >
>
> >  ---- abbie
>
> >  SOAP security involves header = extensions that allow for 
> > Authorization/Authentication information = to be exchaged (including
> > info for
> ppp>  XML encryption/Sig)
> >  There are also provision for Policy = exchange, Privacy
> exchange, Trust
> > and  routing.
>
> >  In SOAP, you can use routing = information to state which SOAP Actor
> > can be  involved in the data path = etc. This help OPES
> since, since it
> > can define  which callout server can = be used and it can also define
> > what policy to use  etc.. (including = privacy info...)
>
> We have to have the spec and an expert in = implementation in
> order order to appreciate this.
>
>
> >  > >  For OPES to be = usefull, I think it should be able to
> > interoperate
> >  > > with web  services. = At the end of the day, OPES provide a
> >  > service to
> >  > > the data  = provider/consumer application.
> >  >
> >  > Can you tell us what it means = to interoperate with web
> >  > services?  I'm not against = it, but I've got only the vaguest
> >  > notion of what it might mean = for OPES.
> >  >
>
> >  ---- ABBIE
> >  WHat I mean here, is that the = service provider that deploy OPES,
> > should be  able to offer services as = they become avilable
> and as long
> > as they make good  financial sence. = Since off hand you do not know
> > what a killer APP is, the  OPES = platform should be
> extendale, meaning,
> > the OPES provider should be able  to = offer a service at a minimal
> > cost. SO if SOAP ids the callout = protocol,  and the new
> service ids a
> > web service, implementiung it in OPES is = quick and  strighat fwd.
> > Otherwise, u need to customise the service = at leat at the OPES  end.
>
> >  WOrst case secnario, the actual = provider of the service
> need to TALK
> > OPES.
>
> What does that mean?
>
> >  > >  I know that SOAP has = been looked at before by OPES. As a
> >  > start, do we
> >  > > have a  pointer on = the previous work, if yes can someone
> >  > please advice
> >  > > where we can  get = it.
> >  >
> >  > My memory of this is that Lee = Rafalow was a strong advocate
> >  > of looking at SOAP, but there = was no work beyond that
> >  > advocacy.  Perhaps others = have done work since then, but have
> >  > not reported it to OPES because = we were concentrating only on
> >  > the initial documents.  We = need to spread the word that we
> >  > are now in a new WG phase and = it is safe to come back to the
> >  > mailing list.
> >  >
> >  > Hilarie
>
>
> >  I have access to that info, but i = thought that there was a
> talk about
> >  comparing SOAP, ICAP etc.. that was = presented in one of
> the BOFs meetings.
>
> Don't remember it, but the original website has = been consumed.
>
> >  abbie
>
>
>
> >  >
>
>

------_=_NextPart_001_01C2D60D.89070414-- From owner-ietf-openproxy@mail.imc.org Sun Feb 16 19:04:13 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA01365 for ; Sun, 16 Feb 2003 19:04:12 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1GNemY02427 for ietf-openproxy-bks; Sun, 16 Feb 2003 15:40:48 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1GNekd02423 for ; Sun, 16 Feb 2003 15:40:47 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1H0CWnT004032; Sun, 16 Feb 2003 17:12:33 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1GNfQa03059; Sun, 16 Feb 2003 16:41:26 -0700 Date: Sun, 16 Feb 2003 16:41:26 -0700 Message-Id: <200302162341.h1GNfQa03059@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: abbieb@nortelnetworks.com Cc: ietf-openproxy@imc.org In-reply-to: Yourmessage <87609AFB433BD5118D5E0002A52CD75404E0C102@zcard0k6.ca.nortel.com> Subject: RE: Start on OPES protocol work Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Thanks for the explanation of what happened to the BOF-era material; my bookmarks to it no longer work, so it was confusing. I see that it is on the new webpage, under "non WG". It's really a shame, though, that it wasn't possible to make the ietf-opes.org domain work transparently. Overall, I think that in a practical sense, interoperability on the Internet is being badly eroded, but that's a rant for another place. I see that we already have a draft about using OPES and web services in that non WG area: http://standards.nortelnetworks.com/opes/non-wg-doc/draft-rrahman-web-service-00.txt I think it probably serves as a reasonable starting point for the part of the discussion, though it seems long on format and short on motivation. It also seems to be proposing extensions to SOAP in order to provide random access to message parts. Is that right - SOAP messages don't normally use a manifest? Hilarie From owner-ietf-openproxy@mail.imc.org Sun Feb 16 21:12:31 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA02417 for ; Sun, 16 Feb 2003 21:12:30 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1H1pjf04828 for ietf-openproxy-bks; Sun, 16 Feb 2003 17:51:45 -0800 (PST) Received: from zsc3s004.nortelnetworks.com (h65s138a81n47.user.nortelnetworks.com [47.81.138.65]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1H1phd04824 for ; Sun, 16 Feb 2003 17:51:44 -0800 (PST) Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zsc3s004.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1H1pbV27448; Sun, 16 Feb 2003 17:51:38 -0800 (PST) Received: from zsc3c026.us.nortel.com ([47.81.138.26]) by zsc3c028.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1LWHQ78N; Sun, 16 Feb 2003 17:51:37 -0800 Received: from DD59WS11 (artpt5tv.us.nortel.com [47.140.52.245]) by zsc3c026.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1DD9295S; Sun, 16 Feb 2003 17:51:36 -0800 Message-ID: <002901c2d627$24eb1550$f5348c2f@DD59WS11> X-Sybari-Space: 00000000 00000000 00000000 From: "Reinaldo Penno" To: "Martin Stecher" , "OPES Group" References: <72992B39BBD9294BB636A960E89AE02E8FD1C6@hermes.webwasher.com> Subject: Re: Request/response pairs Date: Sun, 16 Feb 2003 20:51:51 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit Request/response pairs ----- Original Message ----- From: Martin Stecher To: OPES Group Sent: Sunday, February 16, 2003 12:33 PM Subject: Request/response pairs Hi. Yet two other questions regarding OPES request/response pairs: - Should the protocol require that there is always a response to each request? {RP}No, not at all, but I guess this is laid out on the protocol requirements, isn't it? While writing the ICAP extension draft, I was informed about an additional method for logging that has been introduced. This method sends only "requests" (better notifications) to the server (to give a REQMOD service information about the downloaded object from a former REQMOD request for logging purpose). This method does not expect any response. Are these messages w/o response something the OPES protocol should consider or allow or forbid explicitly? - While connections MUST only initiated from client to server, what is about requests on that connection? May it be useful to allow the callout server to send a request back to the client? {RP} well, requests back would be a response, isn't it? But I agree that inside a connection the callout server can issue requests to the opes processor. Keep alive is one example, various statistics is another, fail-over is another. I am thinking here about the keep-alive messages? It is not only in the interest of the OPES device to check whether the server is still alive. The callout server could also be concerned that no new request is coming along an open connection and wants to verify that the client is not down. What are your thoughts on this? Regards Martin From owner-ietf-openproxy@mail.imc.org Sun Feb 16 21:24:59 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA02518 for ; Sun, 16 Feb 2003 21:24:57 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1H26L705203 for ietf-openproxy-bks; Sun, 16 Feb 2003 18:06:21 -0800 (PST) Received: from zsc3s004.nortelnetworks.com (h65s138a81n47.user.nortelnetworks.com [47.81.138.65]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1H26Kd05199 for ; Sun, 16 Feb 2003 18:06:20 -0800 (PST) Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zsc3s004.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1H26EV27607; Sun, 16 Feb 2003 18:06:15 -0800 (PST) Received: from zsc3c026.us.nortel.com ([47.81.138.26]) by zsc3c028.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1LWHQ8A7; Sun, 16 Feb 2003 18:06:15 -0800 Received: from DD59WS11 (artpt5tv.us.nortel.com [47.140.52.245]) by zsc3c026.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1DD9295Y; Sun, 16 Feb 2003 18:06:14 -0800 Message-ID: <004e01c2d629$30201f90$f5348c2f@DD59WS11> X-Sybari-Space: 00000000 00000000 00000000 From: "Reinaldo Penno" To: "Martin Stecher" , "OPES Group" References: <72992B39BBD9294BB636A960E89AE02E50C100@hermes.webwasher.com> Subject: Re: Start on OPES protocol work Date: Sun, 16 Feb 2003 21:06:29 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit ----- Original Message ----- From: "Martin Stecher" To: "OPES Group" Sent: Sunday, February 16, 2003 7:57 AM Subject: Re: Start on OPES protocol work > > > > > > Defining it this way should guarantee a very strict and > > clear protocol > > > message syntax that avoids those complicated phrases like > > "after sending > > > this, the client has to wait until blah blah" and this > > should then have > > > a positive effect on protocol implementations that should > > avoid that > > > messages on an open persistent connection run out-of sync. > > > > "out-of sync"? I'm not sure what the problem is. Could you > > elaborate, > > because it's important that we all understand what the > > requirements mean. > > With "out-of-sync" I mean that either no more request can be sent on an open connection because client and server are waiting endless time for an event that the other peer will not trigger (again) or a connection shutdown because one peer detected a syntax error by peeking into a wrong part of the data. > > I know from ICAP that these things could happen. Two examples: > > Due to the preview feature and the 204 response an ICAP server wants to stop a message transaction early; it sends the 204 response and is prepared for the next request that comes on this connection. Unfortunately it was too lazy in being patient to wait and acknowledge the rest of the data from the first message that it does not longer care about but the ICAP client is still forwarding. The ICAP server tries to interpret that later part of the message data as the next ICAP request header and fails: Server is not longer in sync. Hello Martin, I guess that a server could also end a transaction early for security reasons...say, wouldn't sequence numbers + handshake + timers policy ala TCP take care of this. It seems to me ( if I captured your thoughts right) kind of the same problem we have in TCP when one end closes a connection and there are still packets in flight. regards, Reinaldo > > Or another example from ICAP: > Due to the little complicated "ieos extension" indicating that a message is not larger than the preview which was sent, an ICAP server could be buggy in a way that waits for more data which is not coming. ICAP client and server wait endless time for each other. I saw an implementation where this happened only for the case in which HTTP body size was exactly the size of the preview. > > > > > > > 2. > > > The same question from Markus points me to another feature > > that got on > > > my wish list last year: Although I know that we want to > > concentrate on > > > HTTP, section 3.11 encourages us to define a protocol that > > could be used > > > for any application-layer protocol. And unlike most application > > > protocols that have one source and one destination, SMTP > > has one sender > > > but many recipients. An OPES protocol that is prepared to > > encapsulate > > > SMTP messages should be aware that a callout service may > > have different > > > results for different recipients. > > > Therefore I'd welcome if the OPES protocol is able (or can > > be easily > > > extended) to send multiple replies on one request. > > > > Hmmmm - I'm not sure that there's a clear need for multiple replies, > > interesting as it is. Is this example for OPES located between the > > sender's mail agent and the SMTP server? Or for OPES located between > > an SMTP server and the users, working on a message sent to a local > > alias that is a list? Somehow the callout server (or the OPES box?) > > knows the expansion of the alias? > > This example is for an OPES box that works as an intermediate SMTP gateway. It could be the sender's SMTP server itself ot a later gateway in the forwarding chain. > If this box receives message for 20 users of a single domain, it is a bad idea to create 20 copies of the message first and sending one-sender-one-recipient-pairs to the callout service when maybe 18 copies are filtered with the same settings but 2 copies have a different filter result. > I'd would like a protocol that allows the box to forward the SMTP message as is and the callout server to reply "Here is a modified message for 18 recipients and a second message copy for the other two recipients following right after." > > [...] > > > > > > I think about a very lightweight but still powerfull and flexible > > > protocol, building on components of HTTP/HTTPS. > > > Only knowing a little about SOAP and BEEP, I think that > > they are too > > > complex to define a direct base for the OPES protocol, > > although there > > > seem to be very interesting concepts which could be integrated. > > > > SOAP might be considered complicated, but BEEP seems to be an entirely > > different and simpler kind of thing. Possibly simpler than HTTP. > > I would like an even simpler basic message structure. Please see my argumentation in my response to Robert. > > > Martin > From owner-ietf-openproxy@mail.imc.org Sun Feb 16 22:01:12 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA03172 for ; Sun, 16 Feb 2003 22:01:11 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1H2SvG05854 for ietf-openproxy-bks; Sun, 16 Feb 2003 18:28:57 -0800 (PST) Received: from zrc2s0jx.nortelnetworks.com (zrc2s0jx.nortelnetworks.com [47.103.122.112]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1H2Std05850 for ; Sun, 16 Feb 2003 18:28:55 -0800 (PST) Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1H2SnU03433; Sun, 16 Feb 2003 20:28:50 -0600 (CST) Received: from zsc3c026.us.nortel.com ([47.81.138.26]) by zsc3c028.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1LWHQ8FC; Sun, 16 Feb 2003 18:28:50 -0800 Received: from DD59WS11 (artpt5p5.us.nortel.com [47.140.52.90]) by zsc3c026.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1DD92956; Sun, 16 Feb 2003 18:28:49 -0800 Message-ID: <007301c2d62c$5773b0e0$f5348c2f@DD59WS11> X-Sybari-Space: 00000000 00000000 00000000 From: "Reinaldo Penno" To: "Martin Stecher" , "OPES Group" References: <72992B39BBD9294BB636A960E89AE02E50C0FF@hermes.webwasher.com> Subject: Re: Start on OPES protocol work Date: Sun, 16 Feb 2003 21:28:14 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit Hi, the capability negotiation was in part to avoid having to repeat fields in the packets since the two ends would have agreed in the protocol version and several other parameters. Some other protocols where there is some form of capability negotiation (specially routing protocols) repeat the version number in the packets, but routing is something that doesn't change often (at least in theory) and you can put thousands of routes in one packet, so overhead is not an issue. I see the the callout protocol having a control phase and a data phase. In the control phase the version number will be in the packet (in the header or as another parameter) so that the two ends can agree in the minimum set of semantics. In the data phase where the opes processor is making the callout work for its meal, there is no need to the version number (and others) anymore. Some people might argue that a version number in the packets make debugging easier...but we can also achieve this by having a standardized list of operations (for each protocol version) that a opes processor can request. Of course extensions should go though this WG and normal IETFT process. Just like PPP extensions. what do you think? regards, Reinaldo ----- Original Message ----- From: "Martin Stecher" To: "OPES Group" Sent: Sunday, February 16, 2003 7:15 AM Subject: Re: Start on OPES protocol work > > > > > > 3. > > > I like to minimize the message overhead (see my performance > > > considerations in point 5). Together with what I said above > > in the first > > > section, I think that a message header and footer (if a footer is > > > necessary at all) should be as short as possible. Usually, protocols > > > write a protocol version into each request and response - but why? > > > > IMO, because it reduces the state to be held by the endpoints of the > > connection. > > I guess that some parameters that are exchanged during connection establishment need to be kept anyway; I think that another version field in the state info will not hurt at all. > On the other hand you are certainly right that the version number overhead is not a big issue anyway. I just wanted to give an example. There is usually much more information which is repeated with every request on the same connection. > > > > > > Isn't > > > it enough to do this once when a callout connection is > > established.? All > > > the messages within a persistent connection will be of the > > same version. > > > When we try to keep this in mind, every message could be > > designed to be > > > a minimum around the payload. > > > > > > Messages should also follow a same general syntax to make parsing as > > > simple and fast as possible and to allow to skip easily > > message types > > > that are optional. > > > > Agreed. IMO this has a greater impact on performance than a few bytes > > for version identification. > > Correct. But let me again argue why also the little additions can hurt: > If the OPES client needs to wrap parts of the application message in a structure with parts in the beginning and at the end, this usually means to copy the message data on application layer again to setup the structure. While the OPES client will need to forward huge amount of data to the callout service, reducing the number of message data copies should be helpful. > > > > > The thing to remember is that any general purpose protocol will be > > slower than a specially tuned protocol. We're building a > > general purpose > > tool here - so reliability, ease of implementation and > > interoperability > > are (IMO) greater priorities. > > > > Yes, they are but this does not mean that the protocol has to be slow. If we also look at these things, I guess that we can have only a little distance to specially tuned protocols. > If we fail here the OPES protocol could stay a nice academic experience without relevance to the industry. > > Martin > From owner-ietf-openproxy@mail.imc.org Mon Feb 17 01:23:27 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA05853 for ; Mon, 17 Feb 2003 01:23:26 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1H5xTe10155 for ietf-openproxy-bks; Sun, 16 Feb 2003 21:59:29 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1H5xRd10151 for ; Sun, 16 Feb 2003 21:59:27 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1H5xVeM048733 for ; Sun, 16 Feb 2003 22:59:31 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1H5xVeo048732; Sun, 16 Feb 2003 22:59:31 -0700 (MST) (envelope-from rousskov) Date: Sun, 16 Feb 2003 22:59:31 -0700 (MST) From: Alex Rousskov To: OPES Group Subject: protocol core now, transport/encoding later Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Hello, I would like to suggest that the selection of a particular underlying protocol (HTTP, BEEP, SOAP, etc.) is both not important and premature at this time. It is not important because whatever protocol we come up with would be possible to implement on top of any of those lower-level protocols/APIs. It is premature because particular protocol bindings and even performance would depend on minor protocol details that are yet unknown. The same is probably true about XML-versus-text-versus-binary encoding decision. Any approach will "work". We can pick the "best" later because the selection will not (should not) affect the core protocol design. I would also recommend to ignore things like including/excluding version numbers or connection details from certain messages. Those are just optimizations. There is nothing to optimize yet. We all agree that the protocol should be "simple" and "fast". The following are some of the major decision points that will affect the core, based on the discussion so far. The items are taken from your messages, but I tried to exclude questions that are already causing long discussions while being, IMHO, of secondary importance. The order is semi-random; these are all "major" items. 0) Whether there are clear/strict client-server roles or whether it is possible for the server to send requests to the client. 1) Whether the protocol is based on (request, response) transactions or on possibly isolated messages (i.e., a "request" may not demand/expect a response) 2) Whether a single request/message can solicit or trigger multiple responses (e.g., OPES server expanding an SMTP recipient alias into several addresses and handling those addresses differently for the same OPES request) 3) How exceptions are delivered to the other party (e.g., "I am not interested in seeing any more of this message") 4) How the "preview" feature is handled. What the data-granularity of an OPES data transaction is (whole message, header+tail, arbitrary range, arbitrary collection of bytes?) Did I miss any? It would be nice to have a more-or-less complete list so that one can find/propose a simple protocol covering all decision points. We can then compare proposed solutions and fill in the details. Thank you, Alex. -- | HTTP performance - Web Polygraph benchmark www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite | all of the above - PolyBox appliance From owner-ietf-openproxy@mail.imc.org Mon Feb 17 05:10:59 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA19026 for ; Mon, 17 Feb 2003 05:10:58 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1H9lhT16386 for ietf-openproxy-bks; Mon, 17 Feb 2003 01:47:43 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1H9led16380 for ; Mon, 17 Feb 2003 01:47:41 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id P7NUS5I3; 17 Feb 2003 10:47:34 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: SMTP filtering use case X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Mon, 17 Feb 2003 10:44:32 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C103@hermes.webwasher.com> Thread-Topic: SMTP filtering use case Thread-Index: AcLWaZdyX0pnBhwFSWC3bmYl2pVMLw== From: "Martin Stecher" To: "OPES WG (E-mail)" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1H9lgd16382 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit Hi Hilarie and Alex, I think that we are not talking about the same SMTP filtering scenario. Let me build on the virus filtering example and describe a concrete use case: There is a SMTP MTA dealing as the incoming mail server of a company. It is OPES enabled and uses a callout service to do virus filtering on all the incoming messages. Every individual in the company can select his/her preference: Infected messages can either be replaced by an alert message or can be repaired. This property is kept in a directory service for the company. The OPES SMTP device is a general implementation which is not aware of virus scanning techniques and especially not on the user preferences, it does not access the directory service at all, but the callout service does. Now the OPES device receives an email messsage containing a virus which is sent to 20 recipients within the company. It is a single message! The OPES client forwards this one message to the callout service. That one looks up the AV filter preference in the company's directory service. The callout service detects the virus and sees that 18 recipients want to have the message repaired and 2 recipients prefer to get the virus replaced by an alert. From this point on two copies of the SMTP message are required with different content. One to be sent to 18 recipients the other to the other 2. How could this be solved? - By enabling the OPES protocol to return two copies of an application message in response to one filtering request - By requiring that the OPES edge device ensures to pre-generate copies of the application message where each copy guarantees to result in only a single response. Martin From owner-ietf-openproxy@mail.imc.org Mon Feb 17 05:29:13 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA19224 for ; Mon, 17 Feb 2003 05:29:10 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HA6CR16802 for ietf-openproxy-bks; Mon, 17 Feb 2003 02:06:12 -0800 (PST) Received: from hosting.altserver.com ([209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HA6Bd16798 for ; Mon, 17 Feb 2003 02:06:11 -0800 (PST) Received: from f01m-9-237.d1.club-internet.fr ([212.194.0.237] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18kiA5-0001xc-00 for ietf-openproxy@imc.org; Mon, 17 Feb 2003 02:06:09 -0800 Message-Id: <5.2.0.9.0.20030217105837.03c40c80@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 17 Feb 2003 11:04:41 +0100 To: OPES Group From: jfcm Subject: Re: Start on OPES protocol work In-Reply-To: <007301c2d62c$5773b0e0$f5348c2f@DD59WS11> References: <72992B39BBD9294BB636A960E89AE02E50C0FF@hermes.webwasher.com> Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-47DC1374; boundary="=======27BA51A=======" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=======27BA51A======= Content-Type: text/plain; x-avg-checked=avg-ok-47DC1374; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit On 03:28 17/02/03, Reinaldo Penno said: >Hi, > >the capability negotiation was in part to avoid having to repeat fields in >the packets since the two ends would have agreed in the protocol version and >several other parameters. Some other protocols where there is some form of >capability negotiation (specially routing protocols) repeat the version >number in the packets, but routing is something that doesn't change often >(at least in theory) and you can put thousands of routes in one packet, so >overhead is not an issue. > >I see the the callout protocol having a control phase and a data phase. In >the control phase the version number will be in the packet (in the header or >as another parameter) so that the two ends can agree in the minimum set of >semantics. In the data phase where the opes processor is making the callout >work for its meal, there is no need to the version number (and others) >anymore. > >Some people might argue that a version number in the packets make debugging >easier...but we can also achieve this by having a standardized list of >operations (for each protocol version) that a opes processor can request. Of >course extensions should go though this WG and normal IETFT process. Just >like PPP extensions. > >what do you think? IMHO there should be several layers. 1. the link. Is the service here. What are the general paramenters of the service (version, owner, rates, oerating terms and conditions, etc.). Also is the link in operation (last time of exchange) 2. the connexion. What is the accepted traffic rate, the type of encryption, the accepted protocols, etc. Also is the connexion alive (last time of an exchange) 3. the cession. Who is demanding, who is paying, the defaults to be used, the service priority, etc. Radom key of the pseudo-random ref. Last exchange pseudo-random ref, expected next exchange pseudo-random ref. jfc --=======27BA51A======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-47DC1374 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.454 / Virus Database: 253 - Release Date: 10/02/03 --=======27BA51A=======-- From owner-ietf-openproxy@mail.imc.org Mon Feb 17 05:30:02 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA19241 for ; Mon, 17 Feb 2003 05:29:59 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HA6At16796 for ietf-openproxy-bks; Mon, 17 Feb 2003 02:06:10 -0800 (PST) Received: from hosting.altserver.com ([209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HA68d16791 for ; Mon, 17 Feb 2003 02:06:08 -0800 (PST) Received: from f01m-9-237.d1.club-internet.fr ([212.194.0.237] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18kiA0-0001xc-00 for ietf-openproxy@imc.org; Mon, 17 Feb 2003 02:06:05 -0800 Message-Id: <5.2.0.9.0.20030217104204.03c355f0@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 17 Feb 2003 10:48:53 +0100 To: OPES Group From: jfcm Subject: Re: Request/response pairs In-Reply-To: <72992B39BBD9294BB636A960E89AE02E8FD1C6@hermes.webwasher.co m> Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-47DC1374; boundary="=======BF0183C=======" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=======BF0183C======= Content-Type: multipart/alternative; x-avg-checked=avg-ok-47DC1374; boundary="=====================_5679738==.ALT" --=====================_5679738==.ALT Content-Type: text/plain; x-avg-checked=avg-ok-47DC1374; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit On 18:33 16/02/03, Martin Stecher said: >Hi. >Yet two other questions regarding OPES request/response pairs: >- Should the protocol require that there is always a response to each > request? > > While writing the ICAP extension draft, I was informed about an > additional method for logging that has been introduced. > This method sends only "requests" (better notifications) to the > server (to give a REQMOD service information about the downloaded > object from a former REQMOD request for logging purpose). > This method does not expect any response. The main problem of Internet is the lack of acks. Please let not repeat it in here. But there may be smart schemes to get virtual acks, however we want to avoid being out of sync. I do not see how you can do that except with a timer on an ack if there is no other exchange. > Are these messages w/o response something the OPES protocol should > consider or allow or forbid explicitly? > >- While connections MUST only initiated from client to server, what is > about requests on that connection? > May it be useful to allow the callout server to send a request back > to the client? > > I am thinking here about the keep-alive messages? > It is not only in the interest of the OPES device to check whether > the server is still alive. The callout server could also be concerned > that no new request is coming along an open connection and wants to > verify that the client is not down. This is necessary. But these mesages are different nature. They basically maintain a link with a (virtual) machine. They MUST be initiated by the server (or how the client is to know that the server does exist) either as a point to point or as a boradcast. "hi! clients for service "so and so" I am alive.". Introducing a DNS like on this seems to be a complex move, an OPES must ract very fast saying if it can or not assume he service it will delegate to the server. jfc --=====================_5679738==.ALT Content-Type: text/html; x-avg-checked=avg-ok-47DC1374; charset=us-ascii Content-Transfer-Encoding: 8bit On 18:33 16/02/03, Martin Stecher said:
Hi.
Yet two other questions regarding OPES request/response pairs:
- Should the protocol require that there is always a response to each
  request?
 
  While writing the ICAP extension draft, I was informed about an
  additional method for logging that has been introduced.
  This method sends only "requests" (better notifications) to the
  server (to give a REQMOD service information about the downloaded
  object from a former REQMOD request for logging purpose).
  This method does not expect any response.

The main problem of Internet is the lack of acks. Please let not
repeat it in here. But there may be smart schemes to get virtual acks,
however we want to avoid being out of sync. I do not see how you
can do that except with a timer on an ack if there is no other
exchange.

  Are these messages w/o response something the OPES protocol should
  consider or allow or forbid explicitly?
 
- While connections MUST only initiated from client to server, what is
  about requests on that connection?
  May it be useful to allow the callout server to send a request back
  to the client?
 
  I am thinking here about the keep-alive messages?
  It is not only in the interest of the OPES device to check whether
  the server is still alive. The callout server could also be concerned
  that no new request is coming along an open connection and wants to
  verify that the client is not down.

This is necessary. But these mesages are different nature. They
basically maintain a link with a (virtual) machine. They MUST
be initiated by the server (or how the client is to know that the
server does exist) either as a point to point or as a boradcast.

"hi! clients for service "so and so" I am alive.".

Introducing a DNS like on this seems to be a complex
move, an OPES must ract very fast saying if it can or not assume
he service it will delegate to the server.

jfc

--=====================_5679738==.ALT-- --=======BF0183C======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-47DC1374 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.454 / Virus Database: 253 - Release Date: 10/02/03 --=======BF0183C=======-- From owner-ietf-openproxy@mail.imc.org Mon Feb 17 05:30:33 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA19256 for ; Mon, 17 Feb 2003 05:30:32 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HA6Ge16808 for ietf-openproxy-bks; Mon, 17 Feb 2003 02:06:16 -0800 (PST) Received: from hosting.altserver.com ([209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HA6Fd16804 for ; Mon, 17 Feb 2003 02:06:15 -0800 (PST) Received: from f01m-9-237.d1.club-internet.fr ([212.194.0.237] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18kiA8-0001xc-00 for ietf-openproxy@imc.org; Mon, 17 Feb 2003 02:06:13 -0800 Message-Id: <5.2.0.9.0.20030217110715.03c46cd0@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Mon, 17 Feb 2003 11:10:36 +0100 To: OPES Group From: jfcm Subject: Re: protocol core now, transport/encoding later In-Reply-To: Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-47DC1374; boundary="=======11BE34B0=======" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=======11BE34B0======= Content-Type: text/plain; x-avg-checked=avg-ok-47DC1374; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit At 06:59 17/02/03, Alex Rousskov wrote: > 0) Whether there are clear/strict client-server roles or > whether it is possible for the server to send requests > to the client. > > 1) Whether the protocol is based on (request, response) > transactions or on possibly isolated messages (i.e., a > "request" may not demand/expect a response) > > 2) Whether a single request/message can solicit or > trigger multiple responses (e.g., OPES server expanding > an SMTP recipient alias into several addresses and > handling those addresses differently for the same OPES > request) > > 3) How exceptions are delivered to the other party (e.g., > "I am not interested in seeing any more of this message") > > 4) How the "preview" feature is handled. What > the data-granularity of an OPES data transaction is (whole > message, header+tail, arbitrary range, arbitrary collection > of bytes?) > > >Did I miss any? It would be nice to have a more-or-less complete list >so that one can find/propose a simple protocol covering all decision >points. We can then compare proposed solutions and fill in the May I suggest that the OPES web master maintains a list on the site where we could list these ideas. Then we could discuss their aggregation into a minimum non conflicting list much more easily? At this stage it could only be to paste them on an Excel page and to give them a number. The target would be first to see the one we really need, the ones wich means the same thing, etc. jfc --=======11BE34B0======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-47DC1374 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.454 / Virus Database: 253 - Release Date: 10/02/03 --=======11BE34B0=======-- From owner-ietf-openproxy@mail.imc.org Mon Feb 17 06:05:44 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA19939 for ; Mon, 17 Feb 2003 06:05:42 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HAiLT18016 for ietf-openproxy-bks; Mon, 17 Feb 2003 02:44:21 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1HAiJd18012 for ; Mon, 17 Feb 2003 02:44:20 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id SY9JGYLW; 17 Feb 2003 11:44:16 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: Start on OPES protocol work X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Mon, 17 Feb 2003 11:41:14 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C104@hermes.webwasher.com> Thread-Topic: Start on OPES protocol work Thread-Index: AcLV/XCZnyyxJ42uS5mDb5qryOaP3QAbGm1g From: "Martin Stecher" To: "The Purple Streak, Hilarie Orman" Cc: Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1HAiKd18013 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit Hi Hilarie, > > 1. I believe that the protocol requirements stipulate the use of > independent "channels", and those were put in exactly for the the case > you are discussing for preview. If channels don't solve the problem, > we need to know why. draft-ietf-opes-protocol-reqs-03 does not mention the "channels", does it? Till this weekend I was not aware that the "channels" concept already implies that each fragment of an application message is sent in its own callout message via the channel. Is this correct? If yes, I appreciate this concept very much and I think it is exactly what I thought about solving the out-of-sync and preview problems. > > 2. For the SMTP case, let's suppose that the site policy > requires virus > scanning, but some subscribers to the OPES box want potential viruses > removed, and some simply want them flagged. Assume a message has been > sent to several recipients of a domain with an OPES virus scanning > service, and its body is denoted as "A" (identified by its hash, I > suppose). If it's the OPES processor that holds the user preferences, > then it can handle the process of sending A to a callout server for > virus scanning and asking for both removal and flagging, possibly in > one request, but at worst in two. Ah, I see, you'd like to eliminate > the two requests, and have a request type that asks for multiple > services and multiple replies. This is different from asking for > multiple services and one reply. A notation for this might > be something > like: > > Request(data=D; service=A AND service=B) > Reply(data=A(B(D))) > > Request(data=D; service=A; service=B) > Reply(data=A(D); data=B(D)) > > Have I captured the requirement correctly? I am not sure. Please have a look into my posting "SMTP filtering use case". > > 3. I know the arguments for simple formats, but please give something > specific about your objection to BEEP. My objection to XML encoding > is that it requires many instructions to parse it into tag pairs and > parameters, to create a parse tree, and to traverse the tree. When > comparing this to direct indexing, or even scans for end-of-header, > end-of-line, XML is very slow. But, experience has shown that there > are distinct limits to what can be accomplished with fixed parameters > positions and bit encodings, and even TCP has had a great deal of > trouble staying within its header limitations. Up at the > application layer, > I think some compromise will be necessary. I'd thought BEEP > was actually > a good point to begin looking for that compromise point. What is so > awful about it? I am really sorry that I started to name protocols. I didn't want to get into this discussion now. On the other hand it shows that existing protocols have interesting concepts that are very worth to be known and understood while discussing OPES protocol needs and I doubt that I would really look in the protocol's details without this discussion. Unfortunately I had only a brief look into RFC 3080 (BEEP core) before sending my first message last week. And in there I found all examples full of XML structure, always repeating same Content-Type header with every single small message and that little PASCAL reminding "END" literal. So on my first glance it did not look very efficient to me. I thought that if each application message's fragment is encapsulated in its own BEEP message this would be a big overhead. Seems that this is not so. By learning a little more about it over the weekend and starting to understand the "channels" concept, I agree that BEEP is very worth to be considered. It is not "awful" (and I never said so). But still we should not elect the one or other protocol now. What we rather need is a common understanding about the best concepts and how they could fulfill the requirements and solve the questions and problems that we list. What other BEEP documents than RFC 3080 should be studied? Is there a good summary about the concepts that could help OPES? If not, could somebody compile one? Martin From owner-ietf-openproxy@mail.imc.org Mon Feb 17 06:28:16 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA20316 for ; Mon, 17 Feb 2003 06:28:15 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HB8aS19190 for ietf-openproxy-bks; Mon, 17 Feb 2003 03:08:36 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HB8ad19186 for ; Mon, 17 Feb 2003 03:08:36 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1HB8TU04758; Mon, 17 Feb 2003 06:08:29 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6T6AD>; Mon, 17 Feb 2003 06:08:29 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E0C15F@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: "The Purple Streak, Hilarie Orman" Cc: ietf-openproxy@imc.org Subject: RE: Start on OPES protocol work Date: Mon, 17 Feb 2003 06:08:27 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D674.E5C374DC" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D674.E5C374DC Content-Type: text/plain hilarie, if u fo to the www.ietf-opes.org it will direct u to the new site (transparently). agree, it serves as a reasonable starting point. The focus of the work was how to SOAP an OPES callout protocol (extensions were proposed over HTTP). However, our approach will be different if we use SOAP as the callout protocol. abbie > -----Original Message----- > From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu] > Sent: Sunday, February 16, 2003 6:41 PM > To: Barbir, Abbie [CAR:1A00:EXCH] > Cc: ietf-openproxy@imc.org > Subject: RE: Start on OPES protocol work > > > Thanks for the explanation of what happened to the BOF-era > material; my bookmarks to it no longer work, so it was > confusing. I see that it is on the new webpage, under "non > WG". It's really a shame, though, that it wasn't possible to > make the ietf-opes.org domain work transparently. Overall, I > think that in a practical sense, interoperability on the > Internet is being badly eroded, but that's a rant for another place. > > I see that we already have a draft about using OPES and web > services in that non WG area: > http://standards.nortelnetworks.com/opes/non-wg-doc/draft-rrah man-web-service-00.txt I think it probably serves as a reasonable starting point for the part of the discussion, though it seems long on format and short on motivation. It also seems to be proposing extensions to SOAP in order to provide random access to message parts. Is that right - SOAP messages don't normally use a manifest? Hilarie ------_=_NextPart_001_01C2D674.E5C374DC Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: Start on OPES protocol work

hilarie,

if u fo to the www.ietf-opes.org it will direct u to = the new site (transparently).

agree, it serves as a reasonable starting = point.
The focus of the work was how to SOAP an OPES = callout protocol (extensions were proposed over HTTP). However, our = approach will be different if we use SOAP as the callout = protocol.


abbie


> -----Original Message-----
> From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu]
> Sent: Sunday, February 16, 2003 6:41 PM
> To: Barbir, Abbie [CAR:1A00:EXCH]
> Cc: ietf-openproxy@imc.org
> Subject: RE: Start on OPES protocol work
>
>
> Thanks for the explanation of what happened to = the BOF-era
> material; my bookmarks to it no longer work, so = it was
> confusing.  I see that it is on the new = webpage, under "non
> WG".  It's really a shame, though, = that it wasn't possible to
> make the ietf-opes.org domain work = transparently. Overall, I
> think that in a practical sense, = interoperability on the
> Internet is being badly eroded, but that's a = rant for another place.
>
> I see that we already have a draft about using = OPES and web
> services in that non WG area:
> http://standards.nortelnetworks.com/opes/non-wg-doc/dr= aft-rrah
man-web-service-00.txt

I think it probably serves as a reasonable starting = point for the part of the discussion, though it seems long on format = and short on motivation. It also seems to be proposing extensions to = SOAP in order to provide random access to message parts.  Is that = right - SOAP messages don't normally use a manifest?

Hilarie

------_=_NextPart_001_01C2D674.E5C374DC-- From owner-ietf-openproxy@mail.imc.org Mon Feb 17 06:28:25 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA20332 for ; Mon, 17 Feb 2003 06:28:24 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HB3M818898 for ietf-openproxy-bks; Mon, 17 Feb 2003 03:03:22 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1HB3Hd18890 for ; Mon, 17 Feb 2003 03:03:17 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id QC62IQYG; 17 Feb 2003 12:03:14 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: Start on OPES protocol work X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Mon, 17 Feb 2003 12:00:12 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C105@hermes.webwasher.com> Thread-Topic: Start on OPES protocol work Thread-Index: AcLWKL6Ujcdp/NKYTcKbqqTkN/cokwASVCSA From: "Martin Stecher" To: "Reinaldo Penno" , "OPES Group" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1HB3Id18895 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit > > Hello Martin, > > I guess that a server could also end a transaction early for security > reasons...say, wouldn't sequence numbers + handshake + timers > policy ala TCP > take care of this. It seems to me ( if I captured your > thoughts right) kind > of the same problem we have in TCP when one end closes a > connection and > there are still packets in flight. > > regards, > > Reinaldo Hi Reinaldo, the sequence number helps if we can also settle these requirments: - each application message's fragment comes with its own sequence number - there is a way to determine that a new sequence for a new callout transaction on the same callout connection starts, e.g. by also assigning a second sequence number to callout transactions. (This is the "channels" concept? Or didn't I get it still?) The important difference to TCP is that we want to finish a transaction but keep the connection alive. Regards Martin From owner-ietf-openproxy@mail.imc.org Mon Feb 17 06:29:51 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA20399 for ; Mon, 17 Feb 2003 06:29:49 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HB0p318711 for ietf-openproxy-bks; Mon, 17 Feb 2003 03:00:51 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HB0od18707 for ; Mon, 17 Feb 2003 03:00:50 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1HB0fU04452; Mon, 17 Feb 2003 06:00:41 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6T577>; Mon, 17 Feb 2003 06:00:41 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E0C15D@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: jfcm , OPES Group Subject: RE: protocol core now, transport/encoding later Date: Mon, 17 Feb 2003 06:00:40 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D673.CED9DC44" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D673.CED9DC44 Content-Type: text/plain hi, I will come up with a list this week, it will be a summary of the results. abbie > -----Original Message----- > From: jfcm [mailto:info@utel.net] > Sent: Monday, February 17, 2003 5:11 AM > To: OPES Group > Subject: Re: protocol core now, transport/encoding later > > > At 06:59 17/02/03, Alex Rousskov wrote: > > 0) Whether there are clear/strict client-server roles or > > whether it is possible for the server to send requests > > to the client. > > > > 1) Whether the protocol is based on (request, response) > > transactions or on possibly isolated messages (i.e., a > > "request" may not demand/expect a response) > > > > 2) Whether a single request/message can solicit or > > trigger multiple responses (e.g., OPES server expanding > > an SMTP recipient alias into several addresses and > > handling those addresses differently for the same OPES > > request) > > > > 3) How exceptions are delivered to the other party (e.g., > > "I am not interested in seeing any more of this message") > > > > 4) How the "preview" feature is handled. What > > the data-granularity of an OPES data transaction > is (whole > > message, header+tail, arbitrary range, arbitrary > collection > > of bytes?) > > > > > >Did I miss any? It would be nice to have a more-or-less > complete list > >so that one can find/propose a simple protocol covering all decision > >points. We can then compare proposed solutions and fill in the > > May I suggest that the OPES web master maintains a list on > the site where > we could list these ideas. Then we could discuss their > aggregation into a > minimum non conflicting list much more easily? > > At this stage it could only be to paste them on an Excel page > and to give > them a number. The target would be first to see the one we > really need, the > ones wich means the same thing, etc. > jfc > > > ------_=_NextPart_001_01C2D673.CED9DC44 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: protocol core now, transport/encoding later

hi,

I will come up with a list this week, it will be a = summary of the results.

abbie


> -----Original Message-----
> From: jfcm [mailto:info@utel.net]
> Sent: Monday, February 17, 2003 5:11 AM
> To: OPES Group
> Subject: Re: protocol core now, = transport/encoding later
>
>
> At 06:59 17/02/03, Alex Rousskov wrote:
> = >         0) Whether there = are clear/strict client-server roles or
> = >            = whether it is possible for the server to send requests
> = >            = to the client.
> >
> = >         1) Whether the = protocol is based on (request, response)
> = >            = transactions or on possibly isolated messages (i.e., a
> = >            = "request" may not demand/expect a response)
> >
> = >         2) Whether a = single request/message can solicit or
> = >            = trigger multiple responses (e.g., OPES server expanding
> = >            = an SMTP recipient alias into several addresses and
> = >            = handling those addresses differently for the same OPES
> = >            = request)
> >
> = >         3) How exceptions = are delivered to the other party (e.g.,
> = >            = "I am not interested in seeing any more of this = message")
> >
> = >         4) How the = "preview" feature is handled. What
> = >            = the data-granularity of an OPES data transaction
> is (whole
> = >            = message, header+tail, arbitrary range, arbitrary
> collection
> = >            = of bytes?)
> >
> >
> >Did I miss any? It would be nice to have a = more-or-less
> complete list
> >so that one can find/propose a simple = protocol covering all decision
> >points. We can then compare proposed = solutions and fill in the
>
> May I suggest that the OPES web master = maintains a list on
> the site where
> we could list these ideas. Then we could = discuss their
> aggregation into a
> minimum non conflicting list much more = easily?
>
> At this stage it could only be to paste them on = an Excel page
> and to give
> them a number. The target would be first to see = the one we
> really need, the
> ones wich means the same thing, etc.
> jfc
>
>
>

------_=_NextPart_001_01C2D673.CED9DC44-- From owner-ietf-openproxy@mail.imc.org Mon Feb 17 11:05:35 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA24699 for ; Mon, 17 Feb 2003 11:05:34 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HFiH227585 for ietf-openproxy-bks; Mon, 17 Feb 2003 07:44:17 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HFiGd27580 for ; Mon, 17 Feb 2003 07:44:16 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1HFiHeM062329 for ; Mon, 17 Feb 2003 08:44:17 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1HFiHD8062328; Mon, 17 Feb 2003 08:44:17 -0700 (MST) (envelope-from rousskov) Date: Mon, 17 Feb 2003 08:44:17 -0700 (MST) From: Alex Rousskov To: "OPES WG (E-mail)" Subject: Re: SMTP filtering use case In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C103@hermes.webwasher.com> Message-ID: References: <72992B39BBD9294BB636A960E89AE02E50C103@hermes.webwasher.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Martin, We are on the same page, I think. I am not sure a single SMTP message can have multiple true recipients because SMTP clients I have seen would usually send multiple copies, one to each To:/Cc: address. (SMTP gurus please correct me). Fan out is possible when recipient is an alias (management@company.com) that SMTP server expands into unfiltered bob@company.com and filtered rob@company.com. As far as I can see, that expansion can happen at the OPES client or at the OPES server (in theory), but the difference is minor. Obviously, this optimization feature complicates the protocol. Special care needs to be taken to insure that all expanded recipients get the message even if, for example, the OPES server goes down in the middle of the processing, after sending the first (marked, unfiltered) response to the first group of recipients. This complicates client-server negotiations: they need to agree on how many responses and to what recipients there will be so that OPES client can take over if OPES server fails (and save the messages to yet unprocessed recipients, for example). Do you think the optimization is worth the complexity? Can you think of other, non-SMTP related uses where multiple responses to a single OPES request would be useful? N.B. A third way to implement what you want is to allow multiple "recipient" addresses in an OPES request to mean that there are actually multiple requests, _without_ duplicating the request on the wire. Same for responses. This approach could be called a "lazy copy" optimization. The difference is subtle, but it causes fewer protocol complications than allowing (at the protocol level) multiple responses to a single request. Thank you, Alex. On Mon, 17 Feb 2003, Martin Stecher wrote: > > Hi Hilarie and Alex, > > I think that we are not talking about the same SMTP filtering > scenario. Let me build on the virus filtering example and describe a > concrete use case: > > There is a SMTP MTA dealing as the incoming mail server of a > company. It is OPES enabled and uses a callout service to do virus > filtering on all the incoming messages. Every individual in the > company can select his/her preference: Infected messages can either > be replaced by an alert message or can be repaired. This property is > kept in a directory service for the company. The OPES SMTP device is > a general implementation which is not aware of virus scanning > techniques and especially not on the user preferences, it does not > access the directory service at all, but the callout service does. > > Now the OPES device receives an email messsage containing a virus > which is sent to 20 recipients within the company. It is a single > message! > > The OPES client forwards this one message to the callout service. > That one looks up the AV filter preference in the company's > directory service. The callout service detects the virus and sees > that 18 recipients want to have the message repaired and 2 > recipients prefer to get the virus replaced by an alert. > > From this point on two copies of the SMTP message are required with > different content. One to be sent to 18 recipients the other to the > other 2. > > How could this be solved? > - By enabling the OPES protocol to return two copies of an application > message in response to one filtering request > - By requiring that the OPES edge device ensures to pre-generate copies > of the application message where each copy guarantees to result in > only a single response. > > > Martin > From owner-ietf-openproxy@mail.imc.org Mon Feb 17 13:47:55 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA28153 for ; Mon, 17 Feb 2003 13:47:54 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HINXQ03838 for ietf-openproxy-bks; Mon, 17 Feb 2003 10:23:33 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HINWd03834 for ; Mon, 17 Feb 2003 10:23:32 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1HItVnT002334; Mon, 17 Feb 2003 11:55:32 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1HIONs19675; Mon, 17 Feb 2003 11:24:23 -0700 Date: Mon, 17 Feb 2003 11:24:23 -0700 Message-Id: <200302171824.h1HIONs19675@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: abbieb@nortelnetworks.com Cc: ietf-openproxy@imc.org In-reply-to: Yourmessage <87609AFB433BD5118D5E0002A52CD75404E0C15F@zcard0k6.ca.nortel.com> Subject: RE: Start on OPES protocol work Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: My bookmarks are of the form: www.ietf-opes.org/index.htm. This does not get transparently redirected, it gets translated to http://standards.nortelnetworks.com/opes/index.htm/index.htm and that gets a 404 error. If Nortel would update the DNS records for ietf-opes.org through the registrar, the domain would work as it did before. Hilarie From owner-ietf-openproxy@mail.imc.org Mon Feb 17 14:05:04 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA28688 for ; Mon, 17 Feb 2003 14:05:03 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HIjgs04441 for ietf-openproxy-bks; Mon, 17 Feb 2003 10:45:42 -0800 (PST) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HIjcd04437 for ; Mon, 17 Feb 2003 10:45:38 -0800 (PST) Received: from oskar3 (h00a0c5e5337d.ne.client2.attbi.com[24.218.184.133]) by rwcrmhc52.attbi.com (rwcrmhc52) with SMTP id <2003021718453505200off2se>; Mon, 17 Feb 2003 18:45:35 +0000 From: "Oskar Batuner" To: "OPES WG \(E-mail\)" Subject: RE: SMTP filtering use case Date: Mon, 17 Feb 2003 13:46:07 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C103@hermes.webwasher.com> Importance: Normal Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit Martin, I believe OPES protocol should work in this kind of scenario by applying it's own transaction boundaries. Being application agnostic callout protocol does not provide one-to-one mapping between callout protocol messages and application messages. In this scenario single OPES transaction may encapsulate multiple response message instances generated by the server. OPES client by its nature is application specific, so in this case it should expect that a single application message included in OPES request may result in multiple application response messages included into the OPES response, together with - again, application-specific - information how to use each application-level response message. Keeping OPES transaction context independent from the application semantics helps to ensure service reliability: OPES client can detect situation when the request processing was not completed and take the appropriate steps. - Oskar > -----Original Message----- > From: owner-ietf-openproxy@mail.imc.org > [mailto:owner-ietf-openproxy@mail.imc.org]On Behalf Of Martin Stecher > Sent: Monday, February 17, 2003 4:45 AM > To: OPES WG (E-mail) > Subject: SMTP filtering use case > > > > Hi Hilarie and Alex, > > I think that we are not talking about the same SMTP filtering scenario. > Let me build on the virus filtering example and describe a concrete > use case: > > There is a SMTP MTA dealing as the incoming mail server of a company. > It is OPES enabled and uses a callout service to do virus filtering on > all the incoming messages. > Every individual in the company can select his/her preference: > Infected messages can either be replaced by an alert message or can be > repaired. > This property is kept in a directory service for the company. > The OPES SMTP device is a general implementation which is not aware of > virus scanning techniques and especially not on the user preferences, > it does not access the directory service at all, but the callout > service does. > > Now the OPES device receives an email messsage containing a virus which > is sent to 20 recipients within the company. It is a single message! > > The OPES client forwards this one message to the callout service. > That one looks up the AV filter preference in the company's directory > service. The callout service detects the virus and sees that 18 > recipients want to have the message repaired and 2 recipients prefer > to get the virus replaced by an alert. > > From this point on two copies of the SMTP message are required with > different content. One to be sent to 18 recipients the other to the > other 2. > > How could this be solved? > - By enabling the OPES protocol to return two copies of an application > message in response to one filtering request > - By requiring that the OPES edge device ensures to pre-generate copies > of the application message where each copy guarantees to result in > only a single response. > > > Martin From owner-ietf-openproxy@mail.imc.org Mon Feb 17 14:05:51 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA28731 for ; Mon, 17 Feb 2003 14:05:49 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HIj6i04411 for ietf-openproxy-bks; Mon, 17 Feb 2003 10:45:06 -0800 (PST) Received: from zrc2s0jx.nortelnetworks.com (zrc2s0jx.nortelnetworks.com [47.103.122.112]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HIj4d04404 for ; Mon, 17 Feb 2003 10:45:05 -0800 (PST) Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1HIis209160; Mon, 17 Feb 2003 12:44:54 -0600 (CST) Received: by zsc3c028.us.nortel.com with Internet Mail Service (5.5.2653.19) id <1LWHRV6Q>; Mon, 17 Feb 2003 10:44:54 -0800 Message-ID: <0A11633F61BD9F40B43ABCC694004F93F18D9A@zsc3c026.us.nortel.com> From: "Reinaldo Penno" To: "'Alex Rousskov'" , "OPES WG (E-mail)" Subject: RE: SMTP filtering use case Date: Mon, 17 Feb 2003 10:44:54 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D6B4.A91528CE" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D6B4.A91528CE Content-Type: text/plain -----Original Message----- From: Alex Rousskov [mailto:rousskov@measurement-factory.com] Sent: Monday, February 17, 2003 10:44 AM To: OPES WG (E-mail) Subject: Re: SMTP filtering use case Martin, We are on the same page, I think. I am not sure a single SMTP message can have multiple true recipients because SMTP clients I have seen would usually send multiple copies, one to each To:/Cc: address. (SMTP gurus please correct me). Fan out is possible when recipient is an alias (management@company.com) that SMTP server expands into unfiltered bob@company.com and filtered rob@company.com. As far as I can see, that expansion can happen at the OPES client or at the OPES server (in theory), but the difference is minor. Obviously, this optimization feature complicates the protocol. Special care needs to be taken to insure that all expanded recipients get the message even if, for example, the OPES server goes down in the middle of the processing, after sending the first (marked, unfiltered) response to the first group of recipients. This complicates client-server negotiations: they need to agree on how many responses and to what recipients there will be so that OPES client can take over if OPES server fails (and save the messages to yet unprocessed recipients, for example). Do you think the optimization is worth the complexity? Can you think of other, non-SMTP related uses where multiple responses to a single OPES request would be useful? [RP] Hummm...I guess whenever the subscriber identity/preferences are blurred by network devices/applications, i.e, you have a M:N relationship between recipients of a message/packet and services. Wouldn't this happen with RTP streaming reflector and multicast? Regards, Reinaldo N.B. A third way to implement what you want is to allow multiple "recipient" addresses in an OPES request to mean that there are actually multiple requests, _without_ duplicating the request on the wire. Same for responses. This approach could be called a "lazy copy" optimization. The difference is subtle, but it causes fewer protocol complications than allowing (at the protocol level) multiple responses to a single request. Thank you, Alex. On Mon, 17 Feb 2003, Martin Stecher wrote: > > Hi Hilarie and Alex, > > I think that we are not talking about the same SMTP filtering > scenario. Let me build on the virus filtering example and describe a > concrete use case: > > There is a SMTP MTA dealing as the incoming mail server of a company. > It is OPES enabled and uses a callout service to do virus filtering on > all the incoming messages. Every individual in the company can select > his/her preference: Infected messages can either be replaced by an > alert message or can be repaired. This property is kept in a directory > service for the company. The OPES SMTP device is a general > implementation which is not aware of virus scanning techniques and > especially not on the user preferences, it does not access the > directory service at all, but the callout service does. > > Now the OPES device receives an email messsage containing a virus > which is sent to 20 recipients within the company. It is a single > message! > > The OPES client forwards this one message to the callout service. That > one looks up the AV filter preference in the company's directory > service. The callout service detects the virus and sees that 18 > recipients want to have the message repaired and 2 recipients prefer > to get the virus replaced by an alert. > > From this point on two copies of the SMTP message are required with > different content. One to be sent to 18 recipients the other to the > other 2. > > How could this be solved? > - By enabling the OPES protocol to return two copies of an application > message in response to one filtering request > - By requiring that the OPES edge device ensures to pre-generate copies > of the application message where each copy guarantees to result in > only a single response. > > > Martin > ------_=_NextPart_001_01C2D6B4.A91528CE Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: SMTP filtering use case

-----Original Message-----
From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
Sent: Monday, February 17, 2003 10:44 AM
To: OPES WG (E-mail)
Subject: Re: SMTP filtering use case



Martin,

        We are on = the same page, I think. I am not sure a single SMTP message can have = multiple true recipients because SMTP clients I have seen would usually = send multiple copies, one to each To:/Cc: address. (SMTP gurus please = correct me). Fan out is possible when recipient is an alias = (management@company.com) that SMTP server expands into unfiltered = bob@company.com and filtered rob@company.com. As far as I can see, that = expansion can happen at the OPES client or at the OPES server (in = theory), but the difference is minor.

Obviously, this optimization feature complicates the = protocol. Special care needs to be taken to insure that all expanded = recipients get the message even if, for example, the OPES server goes = down in the middle of the processing, after sending the first (marked, = unfiltered) response to the first group of recipients. This complicates = client-server negotiations: they need to agree on how many responses = and to what recipients there will be so that OPES client can take over = if OPES server fails (and save the messages to yet unprocessed = recipients, for example).

Do you think the optimization is worth the = complexity? Can you think of other, non-SMTP related uses where = multiple responses to a single OPES request would be useful?

[RP] Hummm...I guess whenever the subscriber = identity/preferences are blurred by network devices/applications, i.e, = you have a M:N relationship between recipients of a message/packet and = services. Wouldn't this happen with RTP streaming reflector and = multicast?

Regards,

Reinaldo


N.B. A third way to implement what you want is to = allow multiple "recipient" addresses in an OPES request to = mean that there are actually multiple requests, _without_ duplicating = the request on the wire. Same for responses. This approach could be = called a "lazy copy" optimization. The difference is subtle, = but it causes fewer protocol complications than allowing (at the = protocol level) multiple responses to a single request.

Thank you,

Alex.


On Mon, 17 Feb 2003, Martin Stecher wrote:

>
> Hi Hilarie and Alex,
>
> I think that we are not talking about the same = SMTP filtering
> scenario. Let me build on the virus filtering = example and describe a
> concrete use case:
>
> There is a SMTP MTA dealing as the incoming = mail server of a company.
> It is OPES enabled and uses a callout service = to do virus filtering on
> all the incoming messages. Every individual in = the company can select
> his/her preference: Infected messages can = either be replaced by an
> alert message or can be repaired. This property = is kept in a directory
> service for the company. The OPES SMTP device = is a general
> implementation which is not aware of virus = scanning techniques and
> especially not on the user preferences, it does = not access the
> directory service at all, but the callout = service does.
>
> Now the OPES device receives an email messsage = containing a virus
> which is sent to 20 recipients within the = company. It is a single
> message!
>
> The OPES client forwards this one message to = the callout service. That
> one looks up the AV filter preference in the = company's directory
> service. The callout service detects the virus = and sees that 18
> recipients want to have the message repaired = and 2 recipients prefer
> to get the virus replaced by an alert.
>
> From this point on two copies of the SMTP = message are required with
> different content. One to be sent to 18 = recipients the other to the
> other 2.
>
> How could this be solved?
> - By enabling the OPES protocol to return two = copies of an application
>   message in response to one = filtering request
> - By requiring that the OPES edge device = ensures to pre-generate copies
>   of the application message where = each copy guarantees to result in
>   only a single response.
>
>
> Martin
>

------_=_NextPart_001_01C2D6B4.A91528CE-- From owner-ietf-openproxy@mail.imc.org Mon Feb 17 14:08:03 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA28784 for ; Mon, 17 Feb 2003 14:08:01 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HIlmF04515 for ietf-openproxy-bks; Mon, 17 Feb 2003 10:47:48 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HIlld04511 for ; Mon, 17 Feb 2003 10:47:47 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1HIlal07165; Mon, 17 Feb 2003 13:47:37 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C64JFD>; Mon, 17 Feb 2003 13:47:36 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E74221@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , OPES Group Subject: RE: protocol core now, transport/encoding later Date: Mon, 17 Feb 2003 13:47:36 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D6B5.09E360DA" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D6B5.09E360DA Content-Type: text/plain Alex, thanks for the input, General remaks, -- good questions that u raise, however, the questions should have been answered in the protocol requirements draft, so i suggest that we start from there. abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Monday, February 17, 2003 1:00 AM > To: OPES Group > Subject: protocol core now, transport/encoding later > > > > Hello, > > I would like to suggest that the selection of a > particular underlying protocol (HTTP, BEEP, SOAP, etc.) is > both not important and premature at this time. It is not > important because whatever protocol we come up with would be > possible to implement on top of any of those lower-level > protocols/APIs. It is premature because particular protocol > bindings and even performance would depend on minor protocol > details that are yet unknown. > > The same is probably true about > XML-versus-text-versus-binary encoding decision. Any approach > will "work". We can pick the "best" later because the > selection will not (should not) affect the core protocol design. > > I would also recommend to ignore things like > including/excluding version numbers or connection details > from certain messages. Those are just optimizations. There is > nothing to optimize yet. We all agree that the protocol > should be "simple" and "fast". > > The following are some of the major decision points > that will affect the core, based on the discussion so far. > The items are taken from your messages, but I tried to > exclude questions that are already causing long discussions > while being, IMHO, of secondary importance. The order is > semi-random; these are all "major" items. > > 0) Whether there are clear/strict client-server roles or > whether it is possible for the server to send requests > to the client. > > 1) Whether the protocol is based on (request, response) > transactions or on possibly isolated messages (i.e., a > "request" may not demand/expect a response) > > 2) Whether a single request/message can solicit or > trigger multiple responses (e.g., OPES server expanding > an SMTP recipient alias into several addresses and > handling those addresses differently for the same OPES > request) > > 3) How exceptions are delivered to the other party (e.g., > "I am not interested in seeing any more of this message") > > 4) How the "preview" feature is handled. What > the data-granularity of an OPES data transaction is (whole > message, header+tail, arbitrary range, arbitrary collection > of bytes?) > > > Did I miss any? It would be nice to have a more-or-less > complete list so that one can find/propose a simple protocol > covering all decision points. We can then compare proposed > solutions and fill in the details. > > > Thank you, > > Alex. > > -- > | HTTP performance - Web > Polygraph benchmark www.measurement-factory.com | HTTP > compliance+ - Co-Advisor test suite > | all of the above - PolyBox appliance > ------_=_NextPart_001_01C2D6B5.09E360DA Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: protocol core now, transport/encoding later

Alex,

thanks for the input,

General remaks,

-- good questions that u raise, however, the = questions should have been answered in the protocol requirements draft, = so i suggest that we start from there.

abbie



> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Monday, February 17, 2003 1:00 AM
> To: OPES Group
> Subject: protocol core now, transport/encoding = later
>
>
>
> Hello,
>
>       I would like to = suggest that the selection of a
> particular underlying protocol (HTTP, BEEP, = SOAP, etc.) is
> both not important and premature at this time. = It is not
> important because whatever protocol we come up = with would be
> possible to implement on top of any of those = lower-level
> protocols/APIs. It is premature because = particular protocol
> bindings and even performance would depend on = minor protocol
> details that are yet unknown.
>
>       The same is = probably true about
> XML-versus-text-versus-binary encoding = decision. Any approach
> will "work". We can pick the = "best" later because the
> selection will not (should not) affect the core = protocol design.
>
> I would also recommend to ignore things like =
> including/excluding version numbers or = connection details
> from certain messages. Those are just = optimizations. There is
> nothing to optimize yet. We all agree that the = protocol
> should be "simple" and = "fast".
>
>       The following = are some of the major decision points
> that will affect the core, based on the = discussion so far.
> The items are taken from your messages, but I = tried to
> exclude questions that are already causing long = discussions
> while being, IMHO, of secondary importance. The = order is
> semi-random; these are all "major" = items.
>
>       0) Whether there = are clear/strict client-server roles or
>          = whether it is possible for the server to send requests
>          to = the client.
>
>       1) Whether the = protocol is based on (request, response)
>          = transactions or on possibly isolated messages (i.e., a
>          = "request" may not demand/expect a response)
>
>       2) Whether a = single request/message can solicit or
>          = trigger multiple responses (e.g., OPES server expanding
>          an = SMTP recipient alias into several addresses and
>          = handling those addresses differently for the same OPES
>          = request)
>
>       3) How = exceptions are delivered to the other party (e.g.,
>          = "I am not interested in seeing any more of this = message")
>
>       4) How the = "preview" feature is handled. What
>          the = data-granularity of an OPES data transaction is (whole
>          = message, header+tail, arbitrary range, arbitrary collection
>          of = bytes?)
>
>
> Did I miss any? It would be nice to have a = more-or-less
> complete list so that one can find/propose a = simple protocol
> covering all decision points. We can then = compare proposed
> solutions and fill in the details.
>
>
> Thank you,
>
> Alex.
>
> --
>          = ;            = ;       | HTTP performance - Web
> Polygraph benchmark www.measurement-factory.com = | HTTP
> compliance+ - Co-Advisor test suite
>          = ;            = ;       | all of the above - PolyBox = appliance
>

------_=_NextPart_001_01C2D6B5.09E360DA-- From owner-ietf-openproxy@mail.imc.org Mon Feb 17 14:23:31 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA29170 for ; Mon, 17 Feb 2003 14:23:30 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HJ2RH05095 for ietf-openproxy-bks; Mon, 17 Feb 2003 11:02:27 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HJ2Qd05091 for ; Mon, 17 Feb 2003 11:02:26 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1HIwhl08307; Mon, 17 Feb 2003 13:58:44 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C64JS4>; Mon, 17 Feb 2003 13:58:43 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E7424F@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: "The Purple Streak, Hilarie Orman" Cc: ietf-openproxy@imc.org Subject: RE: Start on OPES protocol work Date: Mon, 17 Feb 2003 13:58:43 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D6B6.9762CFE4" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D6B6.9762CFE4 Content-Type: text/plain hilarie, yes indeed. abbie > -----Original Message----- > From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu] > Sent: Monday, February 17, 2003 1:24 PM > To: Barbir, Abbie [CAR:1A00:EXCH] > Cc: ietf-openproxy@imc.org > Subject: RE: Start on OPES protocol work > > > My bookmarks are of the form: www.ietf-opes.org/index.htm. > This does not get transparently redirected, it gets > translated to > http://standards.nortelnetworks.com/opes/index> .htm/index.htm > and that gets a 404 error. If Nortel would > update the DNS records for ietf-opes.org through the > registrar, the domain would work as it did before. > > Hilarie > > ------_=_NextPart_001_01C2D6B6.9762CFE4 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: Start on OPES protocol work

hilarie,
yes indeed.

abbie


> -----Original Message-----
> From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu]
> Sent: Monday, February 17, 2003 1:24 PM
> To: Barbir, Abbie [CAR:1A00:EXCH]
> Cc: ietf-openproxy@imc.org
> Subject: RE: Start on OPES protocol work
>
>
> My bookmarks are of the form: = www.ietf-opes.org/index.htm. 
> This does not get transparently redirected, it = gets
> translated to
> http://standards.nortelnetworks.com/opes/index>= .htm/index.htm
> and that gets a 404 error.  If Nortel = would
> update the DNS records for ietf-opes.org = through the
> registrar, the domain would work as it did = before.
>
> Hilarie
>
>

------_=_NextPart_001_01C2D6B6.9762CFE4-- From owner-ietf-openproxy@mail.imc.org Mon Feb 17 14:24:03 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA29191 for ; Mon, 17 Feb 2003 14:24:01 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HJ1fp05067 for ietf-openproxy-bks; Mon, 17 Feb 2003 11:01:41 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HJ1ed05063 for ; Mon, 17 Feb 2003 11:01:41 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1HJ1YU11856; Mon, 17 Feb 2003 14:01:35 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C64JWC>; Mon, 17 Feb 2003 14:01:34 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E74263@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Oskar Batuner , "OPES WG (E-mail)" Subject: RE: SMTP filtering use case Date: Mon, 17 Feb 2003 14:01:33 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D6B6.FCBDE7F2" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D6B6.FCBDE7F2 Content-Type: text/plain Oskar, good point(s) abbie > -----Original Message----- > From: Oskar Batuner [mailto:batuner@attbi.com] > Sent: Monday, February 17, 2003 1:46 PM > To: OPES WG (E-mail) > Subject: RE: SMTP filtering use case > > > > Martin, > > I believe OPES protocol should work in this kind of scenario by > applying it's own transaction boundaries. Being application > agnostic callout protocol does not provide one-to-one mapping > between callout protocol messages and application messages. > > In this scenario single OPES transaction may encapsulate > multiple response message instances generated by the server. > OPES client by its nature is application specific, so > in this case it should expect that a single application message > included in OPES request may result in multiple application > response messages included into the OPES response, together > with - again, application-specific - information how to use > each application-level response message. > > Keeping OPES transaction context independent from the > application semantics helps to ensure service > reliability: OPES client can detect situation when the > request processing was not completed and take the appropriate > steps. > > - Oskar > > > -----Original Message----- > > From: owner-ietf-openproxy@mail.imc.org > > [mailto:owner-ietf-openproxy@mail.imc.org]On Behalf Of > Martin Stecher > > Sent: Monday, February 17, 2003 4:45 AM > > To: OPES WG (E-mail) > > Subject: SMTP filtering use case > > > > > > > > Hi Hilarie and Alex, > > > > I think that we are not talking about the same SMTP filtering > > scenario. Let me build on the virus filtering example and > describe a > > concrete use case: > > > > There is a SMTP MTA dealing as the incoming mail server of > a company. > > It is OPES enabled and uses a callout service to do virus > filtering on > > all the incoming messages. Every individual in the company > can select > > his/her preference: Infected messages can either be replaced by an > > alert message or can be repaired. > > This property is kept in a directory service for the company. > > The OPES SMTP device is a general implementation which is > not aware of > > virus scanning techniques and especially not on the user > preferences, > > it does not access the directory service at all, but the callout > > service does. > > > > Now the OPES device receives an email messsage containing a virus > > which is sent to 20 recipients within the company. It is a single > > message! > > > > The OPES client forwards this one message to the callout > service. That > > one looks up the AV filter preference in the company's directory > > service. The callout service detects the virus and sees that 18 > > recipients want to have the message repaired and 2 > recipients prefer > > to get the virus replaced by an alert. > > > > From this point on two copies of the SMTP message are required with > > different content. One to be sent to 18 recipients the other to the > > other 2. > > > > How could this be solved? > > - By enabling the OPES protocol to return two copies of an > application > > message in response to one filtering request > > - By requiring that the OPES edge device ensures to > pre-generate copies > > of the application message where each copy guarantees to result in > > only a single response. > > > > > > Martin > ------_=_NextPart_001_01C2D6B6.FCBDE7F2 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: SMTP filtering use case

Oskar,

good point(s)

abbie

> -----Original Message-----
> From: Oskar Batuner [mailto:batuner@attbi.com]
> Sent: Monday, February 17, 2003 1:46 PM
> To: OPES WG (E-mail)
> Subject: RE: SMTP filtering use case
>
>
>
> Martin,
>
> I believe OPES protocol should work in this = kind of scenario by
> applying it's own transaction boundaries. Being = application
> agnostic callout protocol does not provide = one-to-one mapping
> between callout protocol messages and = application messages. 
> 
> In this scenario single OPES transaction may = encapsulate
> multiple response message instances generated = by the server.
> OPES client by its nature is application = specific, so
> in this case it should expect that a single = application message
> included in OPES request may result in multiple = application
> response messages included into the OPES = response, together
> with - again, application-specific - = information how to use
> each application-level response message. =
>
> Keeping OPES transaction context independent = from the
> application semantics helps to ensure service =
> reliability: OPES client can detect situation = when the
> request processing was not completed and take = the appropriate
> steps.
>
> - Oskar
>
> > -----Original Message-----
> > From: owner-ietf-openproxy@mail.imc.org =
> > [mailto:owner-ietf-open= proxy@mail.imc.org]On Behalf Of
> Martin Stecher
> > Sent: Monday, February 17, 2003 4:45 = AM
> > To: OPES WG (E-mail)
> > Subject: SMTP filtering use case
> >
> >
> >
> > Hi Hilarie and Alex,
> >
> > I think that we are not talking about the = same SMTP filtering
> > scenario. Let me build on the virus = filtering example and
> describe a
> > concrete use case:
> >
> > There is a SMTP MTA dealing as the = incoming mail server of
> a company.
> > It is OPES enabled and uses a callout = service to do virus
> filtering on
> > all the incoming messages. Every = individual in the company
> can select
> > his/her preference: Infected messages can = either be replaced by an
> > alert message or can be repaired.
> > This property is kept in a directory = service for the company.
> > The OPES SMTP device is a general = implementation which is
> not aware of
> > virus scanning techniques and especially = not on the user
> preferences,
> > it does not access the directory service = at all, but the callout
> > service does.
> >
> > Now the OPES device receives an email = messsage containing a virus
> > which is sent to 20 recipients within the = company. It is a single
> > message!
> >
> > The OPES client forwards this one message = to the callout
> service. That
> > one looks up the AV filter preference in = the company's directory
> > service. The callout service detects the = virus and sees that 18
> > recipients want to have the message = repaired and 2
> recipients prefer
> > to get the virus replaced by an = alert.
> >
> > From this point on two copies of the SMTP = message are required with
> > different content. One to be sent to 18 = recipients the other to the
> > other 2.
> >
> > How could this be solved?
> > - By enabling the OPES protocol to return = two copies of an
> application
> >   message in response to one = filtering request
> > - By requiring that the OPES edge device = ensures to
> pre-generate copies
> >   of the application message = where each copy guarantees to result in
> >   only a single response.
> >
> >
> > Martin
>

------_=_NextPart_001_01C2D6B6.FCBDE7F2-- From owner-ietf-openproxy@mail.imc.org Mon Feb 17 14:45:32 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA29608 for ; Mon, 17 Feb 2003 14:45:31 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HJPNN06282 for ietf-openproxy-bks; Mon, 17 Feb 2003 11:25:23 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HJPLd06278 for ; Mon, 17 Feb 2003 11:25:21 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1HJvMnT004344; Mon, 17 Feb 2003 12:57:22 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1HJQ8q23031; Mon, 17 Feb 2003 12:26:08 -0700 Date: Mon, 17 Feb 2003 12:26:08 -0700 Message-Id: <200302171926.h1HJQ8q23031@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: martin.stecher@webwasher.com Cc: ietf-openproxy@imc.org In-reply-to: Yourmessage <72992B39BBD9294BB636A960E89AE02E50C103@hermes.webwasher.com> Subject: Re: SMTP filtering use case Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Your filtering example is based on delivery to an SMTP server that knows that the recipient of the single message is actually an alias. The OPES service must know the expansion of the alias to the 20 recipients (any one of which could actually be an alias for another list). This means that the SMTP service and the OPES service are very tightly coupled. As I said, I'd always assumed that the OPES processor would have the user preferences, and that's the whole reason for the rules, etc., and for having the OPES processor be the enforcement point for privacy policy. But, I guess that assumption is open to debate. But there's another reason that I'm still not convinced that this SMTP example applies to OPES. If this service and all the user preferences are handled by a callout server, there's no reason that the callout server cannot just forward the transformed SMTP message to a real SMTP server for final delivery. Because SMTP is a store-and-forward service, not end-to-end, this works better than OPES would - there's less I/O involved. Hilarie From owner-ietf-openproxy@mail.imc.org Mon Feb 17 15:15:14 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA00229 for ; Mon, 17 Feb 2003 15:15:13 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HJtjU07168 for ietf-openproxy-bks; Mon, 17 Feb 2003 11:55:45 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1HJtid07164 for ; Mon, 17 Feb 2003 11:55:44 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id ULIRL7T0; 17 Feb 2003 20:55:45 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: AW: SMTP filtering use case X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Mon, 17 Feb 2003 20:52:40 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C10A@hermes.webwasher.com> Thread-Topic: SMTP filtering use case Thread-Index: AcLWuejpLVQ4Bv7uQRCrszrGPJMH3QAAqj5Q From: "Martin Stecher" To: "The Purple Streak, Hilarie Orman" Cc: Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1HJtid07165 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit I have a different view on this. > Your filtering example is based on delivery to an SMTP server that > knows that the recipient of the single message is actually an alias. No, I am not talking about an alias. It is a very common SMTP behavior that one message is sent to multiple recipients, especially if they belong to the same domain. This happens in the SMTP dialog. After the first HELO and MAIL FROM messages the sending MTA sends multiple RCPT TO messages which get all acknowledged one by one. The receiving SMTP MTA keeps the list of all recipients and delivers to all of them (sometimes it needs then to create copies, sometimes not). > The OPES service must know the expansion of the alias to the 20 > recipients (any one of which could actually be an alias for another > list). This means that the SMTP service and the OPES service are very > tightly coupled. As I said, I'd always assumed that the OPES > processor > would have the user preferences, and that's the whole reason for the > rules, etc., and for having the OPES processor be the > enforcement point > for privacy policy. But, I guess that assumption is open to debate. I believe that there will be several implementations of OPES processors that do not want to deal with the complete user preferences and rule engine, but leave this work for the callout service. > > But there's another reason that I'm still not convinced that this SMTP > example applies to OPES. If this service and all the user preferences > are handled by a callout server, there's no reason that the callout > server cannot just forward the transformed SMTP message to a real SMTP > server for final delivery. Because SMTP is a store-and-forward > service, not end-to-end, this works better than OPES would - there's > less I/O involved. This could be done. But we will loose the biggest benefits of the callout strategy at all: Especially for EMail no message must be lost. The OPES processor is the thing that the user trusts, that has long experience what to do with messages that cannot be delivered immediatly, how to manage queues and so on. The callout service is "just" for filtering the messages. If it crashes or fails or has a disk crash, it does not affect all waiting messages. That's a very important point I think. Martin From owner-ietf-openproxy@mail.imc.org Mon Feb 17 15:59:13 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA01282 for ; Mon, 17 Feb 2003 15:59:11 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HKeS008560 for ietf-openproxy-bks; Mon, 17 Feb 2003 12:40:28 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HKeRd08556 for ; Mon, 17 Feb 2003 12:40:27 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1HKeTeM070120 for ; Mon, 17 Feb 2003 13:40:29 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1HKeTK7070119; Mon, 17 Feb 2003 13:40:29 -0700 (MST) (envelope-from rousskov) Date: Mon, 17 Feb 2003 13:40:29 -0700 (MST) From: Alex Rousskov To: OPES Group Subject: RE: protocol core now, transport/encoding later In-Reply-To: <87609AFB433BD5118D5E0002A52CD75404E74221@zcard0k6.ca.nortel.com> Message-ID: References: <87609AFB433BD5118D5E0002A52CD75404E74221@zcard0k6.ca.nortel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Mon, 17 Feb 2003, Abbie Barbir wrote: > -- good questions that u raise, however, the questions should have > been answered in the protocol requirements draft, so i suggest that > we start from there. Abbie, My understanding is that the requirements draft does not answer those questions. Moreover, there is no clear boundary between protocol requirements and actual decision points in protocol design. The requirements may be very specific or quite general. We ended up with general requirements. Fine. The requirements stage is now behind us, and I would like to avoid going back unless absolutely necessary. Regardless of how we answer the questions raised, the resulting protocol is likely to satisfy the requirements draft, and that's all we should care about. It is time for specific protocol-level work, IMO. If my understanding is wrong, and some of the questions are already clearly answered by the requirements draft, please quote these answers, and we will move on. Thank you, Alex. -- | HTTP performance - Web Polygraph benchmark www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite | all of the above - PolyBox appliance From owner-ietf-openproxy@mail.imc.org Mon Feb 17 16:10:54 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA01552 for ; Mon, 17 Feb 2003 16:10:52 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HKqKM08867 for ietf-openproxy-bks; Mon, 17 Feb 2003 12:52:20 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HKqId08863 for ; Mon, 17 Feb 2003 12:52:18 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1HKqLeM070450 for ; Mon, 17 Feb 2003 13:52:21 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1HKqLew070449; Mon, 17 Feb 2003 13:52:21 -0700 (MST) (envelope-from rousskov) Date: Mon, 17 Feb 2003 13:52:20 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: Re: SMTP filtering use case In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C10A@hermes.webwasher.com> Message-ID: References: <72992B39BBD9294BB636A960E89AE02E50C10A@hermes.webwasher.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: It looks like the big question here is about control over user preferences. To me, it seems unreasonable to assume that all user preferences are handled at the OPES processor. It is just too rigid of a model and often very inefficient since the processor knows very little about filtering and the filter (OPES server) may need to know a lot. If we restrict user preference handling to OPES processor, we end up passing a lot of preference information to the OPES server just so that it can process a transaction. Ideally, OPES processor should manage user preferences that affect the decision of what to filter. OPES server should manage user preferences that affect the filtering algorithm. But we should not assume a one-size-fits all scheme. Thus, it looks to me that Martin's example is valid. I am just not yet sure whether the performance optimization is worth the complexity. That is why I asked for more similar uses. Alex. On Mon, 17 Feb 2003, Martin Stecher wrote: > > I have a different view on this. > > > > Your filtering example is based on delivery to an SMTP server that > > knows that the recipient of the single message is actually an alias. > > No, I am not talking about an alias. > It is a very common SMTP behavior that one message is sent to multiple recipients, especially if they belong to the same domain. > This happens in the SMTP dialog. > After the first HELO and MAIL FROM messages the sending MTA sends multiple RCPT TO messages which get all acknowledged one by one. > The receiving SMTP MTA keeps the list of all recipients and delivers to all of them (sometimes it needs then to create copies, sometimes not). > > > The OPES service must know the expansion of the alias to the 20 > > recipients (any one of which could actually be an alias for another > > list). This means that the SMTP service and the OPES service are very > > tightly coupled. As I said, I'd always assumed that the OPES > > processor > > would have the user preferences, and that's the whole reason for the > > rules, etc., and for having the OPES processor be the > > enforcement point > > for privacy policy. But, I guess that assumption is open to debate. > > I believe that there will be several implementations of OPES processors that do not want to deal with the complete user preferences and rule engine, but leave this work for the callout service. > > > > > But there's another reason that I'm still not convinced that this SMTP > > example applies to OPES. If this service and all the user preferences > > are handled by a callout server, there's no reason that the callout > > server cannot just forward the transformed SMTP message to a real SMTP > > server for final delivery. Because SMTP is a store-and-forward > > service, not end-to-end, this works better than OPES would - there's > > less I/O involved. > > This could be done. But we will loose the biggest benefits of the callout strategy at all: > Especially for EMail no message must be lost. The OPES processor is the thing that the user trusts, that has long experience what to do with messages that cannot be delivered immediatly, how to manage queues and so on. > The callout service is "just" for filtering the messages. If it crashes or fails or has a disk crash, it does not affect all waiting messages. That's a very important point I think. > > Martin > From owner-ietf-openproxy@mail.imc.org Mon Feb 17 16:12:00 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA01628 for ; Mon, 17 Feb 2003 16:11:59 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HKqNp08873 for ietf-openproxy-bks; Mon, 17 Feb 2003 12:52:23 -0800 (PST) Received: from brick.recoil.org (brick.recoil.org [194.70.3.130]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1HKqMd08869 for ; Mon, 17 Feb 2003 12:52:22 -0800 (PST) Received: (qmail 16288 invoked by uid 10000); 17 Feb 2003 20:52:12 -0000 Date: Mon, 17 Feb 2003 20:52:12 +0000 From: Anil Madhavapeddy To: Martin Stecher Cc: "The Purple Streak, Hilarie Orman" , ietf-openproxy@imc.org Subject: Re: SMTP filtering use case Message-ID: <20030217205210.GA10753@brick> References: <72992B39BBD9294BB636A960E89AE02E50C10A@hermes.webwasher.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C10A@hermes.webwasher.com> User-Agent: Mutt/1.4i Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Mon, Feb 17, 2003 at 08:52:40PM +0100, Martin Stecher wrote: > > No, I am not talking about an alias. > It is a very common SMTP behavior that one message is sent to multiple > recipients, especially if they belong to the same domain. > This happens in the SMTP dialog. > After the first HELO and MAIL FROM messages the sending MTA sends multiple > RCPT TO messages which get all acknowledged one by one. > The receiving SMTP MTA keeps the list of all recipients and delivers > to all of them (sometimes it needs then to create copies, sometimes not). > I would note that this is an optimization that not all SMTP delivery agents (notably qmail) obey, since it complicates the client implementation too much. RFC2821 (Sec 4.5.4.1) says its a 'SHOULD'. It's also unclear on what to do if there are a huge number of recipients at the same mail server (``a limit MAY be imposed''). It works pretty well in real-life; all servers accept multiple messages in a single transaction, most clients send them that way, but you are still allowed to have lightweight clients which choose to burn some bandwidth for the sake of robustness. > Especially for EMail no message must be lost. The OPES processor > is the thing that the user trusts, that has long experience what to > do with messages that cannot be delivered immediatly, how to manage > queues and so on. If by 'lost' you mean 'silently dropped', then yes. I think it's acceptable for the service to bounce messages back to the sender if there's some kind of unrecoverable transport error, but silently dropping them is very, very bad. And of course, the bounces should be for rare situations! -anil From owner-ietf-openproxy@mail.imc.org Mon Feb 17 17:39:20 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA03030 for ; Mon, 17 Feb 2003 17:39:18 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HMGrq11369 for ietf-openproxy-bks; Mon, 17 Feb 2003 14:16:53 -0800 (PST) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HMGnd11363 for ; Mon, 17 Feb 2003 14:16:49 -0800 (PST) Received: from oskar3 (h00a0c5e5337d.ne.client2.attbi.com[24.218.184.133]) by rwcrmhc52.attbi.com (rwcrmhc52) with SMTP id <2003021722164605200k18ace>; Mon, 17 Feb 2003 22:16:46 +0000 From: "Oskar Batuner" To: "Alex Rousskov" , Subject: RE: SMTP filtering use case Date: Mon, 17 Feb 2003 17:17:18 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 In-Reply-To: Importance: Normal Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit > It looks like the big question here is about control over user > preferences. To me, it seems unreasonable to assume that all user > preferences are handled at the OPES processor. It is just too rigid of > a model and often very inefficient since the processor knows very > little about filtering and the filter (OPES server) may need to know a > lot. If we restrict user preference handling to OPES processor, we end > up passing a lot of preference information to the OPES server just so > that it can process a transaction. I'm afraid we are going too deep with analysis of this specific scenario. If I understand correctly Martin used preferences just as an example of the reason why callout processor may produce multiple responses for a single request. There may be any number of reasons for the callout processor to produce multi-message response, the protocol just SHOULD (MUST?) support multiple responses while preserving transactional integrity. We should not (MUST NOT) impose any limitation on what information is available to the callout processor (including preferences) and how OPES processor and callout processor assign tasks between themselves while performing a collaborative processing - this is application specifics and should be out of scope for protocol design. Oskar > > Ideally, OPES processor should manage user preferences that affect the > decision of what to filter. OPES server should manage user preferences > that affect the filtering algorithm. But we should not assume a > one-size-fits all scheme. > > Thus, it looks to me that Martin's example is valid. I am just not yet > sure whether the performance optimization is worth the complexity. > That is why I asked for more similar uses. > > Alex. > > > On Mon, 17 Feb 2003, Martin Stecher wrote: > > > > > I have a different view on this. > > > > > > > Your filtering example is based on delivery to an SMTP server that > > > knows that the recipient of the single message is actually an alias. > > > > No, I am not talking about an alias. > > It is a very common SMTP behavior that one message is sent to > multiple recipients, especially if they belong to the same domain. > > This happens in the SMTP dialog. > > After the first HELO and MAIL FROM messages the sending MTA sends > multiple RCPT TO messages which get all acknowledged one by one. > > The receiving SMTP MTA keeps the list of all recipients and > delivers to all of them (sometimes it needs then to create copies, > sometimes not). > > > > > The OPES service must know the expansion of the alias to the 20 > > > recipients (any one of which could actually be an alias for another > > > list). This means that the SMTP service and the OPES service are very > > > tightly coupled. As I said, I'd always assumed that the OPES > > > processor > > > would have the user preferences, and that's the whole reason for the > > > rules, etc., and for having the OPES processor be the > > > enforcement point > > > for privacy policy. But, I guess that assumption is open to debate. > > > > I believe that there will be several implementations of OPES > processors that do not want to deal with the complete user > preferences and rule engine, but leave this work for the callout service. > > > > > > > > But there's another reason that I'm still not convinced that this SMTP > > > example applies to OPES. If this service and all the user preferences > > > are handled by a callout server, there's no reason that the callout > > > server cannot just forward the transformed SMTP message to a real SMTP > > > server for final delivery. Because SMTP is a store-and-forward > > > service, not end-to-end, this works better than OPES would - there's > > > less I/O involved. > > > > This could be done. But we will loose the biggest benefits of the > callout strategy at all: > > Especially for EMail no message must be lost. The OPES processor > is the thing that the user trusts, that has long experience what to > do with messages that cannot be delivered immediatly, how to manage > queues and so on. > > The callout service is "just" for filtering the messages. If it > crashes or fails or has a disk crash, it does not affect all > waiting messages. That's a very important point I think. > > > > Martin > > From owner-ietf-openproxy@mail.imc.org Mon Feb 17 18:16:57 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA03719 for ; Mon, 17 Feb 2003 18:16:56 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HMt9w12225 for ietf-openproxy-bks; Mon, 17 Feb 2003 14:55:09 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HMt7d12219 for ; Mon, 17 Feb 2003 14:55:08 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1HNR0nT010886; Mon, 17 Feb 2003 16:27:07 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1HMtCq32349; Mon, 17 Feb 2003 15:55:12 -0700 Date: Mon, 17 Feb 2003 15:55:12 -0700 Message-Id: <200302172255.h1HMtCq32349@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: batuner@attbi.com Cc: ietf-openproxy@imc.org In-reply-to: Yourmessage Subject: RE: SMTP filtering use case Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Mon, 17 Feb 2003 at 17:17:18 -0500 Oskar Batuner avowed, in replying to Alex Rousskov: > > It looks like the big question here is about control over user > > preferences. To me, it seems unreasonable to assume that all user > > preferences are handled at the OPES processor. It is just too rigid of > > a model and often very inefficient since the processor knows very > > little about filtering and the filter (OPES server) may need to know a > > lot. If we restrict user preference handling to OPES processor, we end > > up passing a lot of preference information to the OPES server just so > > that it can process a transaction. > I'm afraid we are going too deep with analysis of this specific scenario. > If I understand correctly Martin used preferences just as an example of > the reason why callout processor may produce multiple responses > for a single request. Discussions that reveal schisms in the meaning of the architecture are very important, whether or not it was the intention of the original message. The lack of specifics to date has been difficult for everyone, and I'm sure that there are more surprises lurking as we discuss a real protocol for real uses. > There may be any number of reasons for the callout processor to > produce multi-message response, the protocol just SHOULD (MUST?) support > multiple responses while preserving transactional integrity. > We should not (MUST NOT) impose any limitation on what > information is available to the callout processor (including > preferences) and how OPES processor and callout processor assign > tasks between themselves while performing a collaborative > processing - this is application specifics and should be > out of scope for protocol design. I think it's not as collaborative and loose and all that. The OPES processor is the enforcement point for privacy. It's designed to be a very fast and fine-grained, policy-based, application-layer switch. If it is not the repository for user preferences, then all the policy enforcement and authentication requirements get pushed to the callout servers. This leaves the OPES processor with almost no function - it might as well be a dumb hardware switch operating at layers 4 through 7. Hilarie From owner-ietf-openproxy@mail.imc.org Mon Feb 17 18:16:59 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA03728 for ; Mon, 17 Feb 2003 18:16:58 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1HMoEC12145 for ietf-openproxy-bks; Mon, 17 Feb 2003 14:50:14 -0800 (PST) Received: from lifelesslap.robertcollins.net (ppp218.adsl88.pacific.net.au [202.7.88.218]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1HMoCd12141 for ; Mon, 17 Feb 2003 14:50:12 -0800 (PST) Received: from localhost ([127.0.0.1] ident=robertc) by lifelesslap.robertcollins.net with esmtp (Exim 3.36 #1 (Debian)) id 18ku4r-0008GS-00; Tue, 18 Feb 2003 09:49:33 +1100 Subject: RE: SMTP filtering use case From: Robert Collins To: Oskar Batuner Cc: Alex Rousskov , ietf-openproxy@imc.org In-Reply-To: References: Content-Type: text/plain Organization: Syncretize Pty Limited Message-Id: <1045522172.31666.3.camel@localhost> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.1 Date: 18 Feb 2003 09:49:32 +1100 Content-Transfer-Encoding: 7bit Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit On Tue, 2003-02-18 at 09:17, Oskar Batuner wrote: > There may be any number of reasons for the callout processor to > produce multi-message response, the protocol just SHOULD (MUST?) support > multiple responses while preserving transactional integrity. Placing this requirement on the protocol without a couple of compelling use cases will add complexity without any gain. I like Alex's cheap-copy approach more than a 1->N request->response relationship. It's easier to predict the behaviour of the former. > We should not (MUST NOT) impose any limitation on what > information is available to the callout processor (including > preferences) and how OPES processor and callout processor assign > tasks between themselves while performing a collaborative > processing - this is application specifics and should be > out of scope for protocol design. All protocols exist to support / enable an application. I'm not suggesting that we should impose limitations - but neither can we design in a vaccuum. To me, if user details are in a directory, detection of user preferences seems to be orthogonal to the OPES transaction. *Identification* of the user within the OPES transaction may be useful though. Rob From owner-ietf-openproxy@mail.imc.org Mon Feb 17 19:22:23 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA04606 for ; Mon, 17 Feb 2003 19:22:21 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1I01Lv13571 for ietf-openproxy-bks; Mon, 17 Feb 2003 16:01:21 -0800 (PST) Received: from crufty.research.bell-labs.com (ns2.research.bell-labs.com [204.178.16.49]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1I01Jd13567 for ; Mon, 17 Feb 2003 16:01:19 -0800 (PST) Received: from grubby.research.bell-labs.com (H-135-104-2-9.research.bell-labs.com [135.104.2.9]) by crufty.research.bell-labs.com (8.12.5/8.12.5) with ESMTP id h1I01KLI033733; Mon, 17 Feb 2003 19:01:21 -0500 (EST) Received: from bronx.dnrc.bell-labs.com (bronx.dnrc.bell-labs.com [135.180.160.8]) by grubby.research.bell-labs.com (8.11.6/8.11.6) with ESMTP id h1I01DY43460; Mon, 17 Feb 2003 19:01:13 -0500 (EST) Received: from bell-labs.com (abeck.lra.lucent.com [192.11.5.153]) by bronx.dnrc.bell-labs.com (8.9.3/8.9.3) with ESMTP id TAA23385; Mon, 17 Feb 2003 19:01:12 -0500 (EST) Message-ID: <3E5177C7.3090704@bell-labs.com> Date: Mon, 17 Feb 2003 19:01:11 -0500 From: Andre Beck User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.3b) Gecko/20030210 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Martin Stecher CC: ietf-openproxy@imc.org Subject: Re: Start on OPES protocol work References: <72992B39BBD9294BB636A960E89AE02E50C104@hermes.webwasher.com> In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C104@hermes.webwasher.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit Hi Martin, > draft-ietf-opes-protocol-reqs-03 does not mention the "channels", does it? The -03 version of the draft refers to them as callout connections (see section 3.4 of the draft). I believe the callout connection concept as described in the draft is very similar to the channel concept in BEEP. -Andre From owner-ietf-openproxy@mail.imc.org Tue Feb 18 05:09:28 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA23971 for ; Tue, 18 Feb 2003 05:09:27 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1I9l0K26077 for ietf-openproxy-bks; Tue, 18 Feb 2003 01:47:00 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1I9kxd26072 for ; Tue, 18 Feb 2003 01:46:59 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1I9kZA18555; Tue, 18 Feb 2003 04:46:36 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C64V5K>; Tue, 18 Feb 2003 04:46:36 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E74645@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Robert Collins , Oskar Batuner Cc: Alex Rousskov , ietf-openproxy@imc.org Subject: RE: SMTP filtering use case Date: Tue, 18 Feb 2003 04:46:35 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D732.9FEF27E4" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D732.9FEF27E4 Content-Type: text/plain > -----Original Message----- > From: Robert Collins [mailto:robert.collins@syncretize.net] > Sent: Monday, February 17, 2003 5:50 PM > To: Oskar Batuner > Cc: Alex Rousskov; ietf-openproxy@imc.org > Subject: RE: SMTP filtering use case > > > > On Tue, 2003-02-18 at 09:17, Oskar Batuner wrote: > > > > There may be any number of reasons for the callout processor to > > produce multi-message response, the protocol just SHOULD > (MUST?) support > > multiple responses while preserving transactional integrity. > SNIP > To me, if user details are in a directory, detection of user > preferences seems to be orthogonal to the OPES transaction. > *Identification* of the user within the OPES transaction may > be useful though. > > Rob SNIP -------- abbie I agree, detection of user preferences is needed. I also believe that Identification of the user within the OPES transaction is more than useful, it may be needed. ------_=_NextPart_001_01C2D732.9FEF27E4 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: SMTP filtering use case

> -----Original Message-----
> From: Robert Collins [mailto:robert.collins@sync= retize.net]
> Sent: Monday, February 17, 2003 5:50 PM
> To: Oskar Batuner
> Cc: Alex Rousskov; = ietf-openproxy@imc.org
> Subject: RE: SMTP filtering use case
>
>
>
> On Tue, 2003-02-18 at 09:17, Oskar Batuner = wrote:
>
>
> > There may be any number of reasons for the = callout processor to
> > produce multi-message response, the = protocol just SHOULD
> (MUST?) support
> > multiple responses while preserving = transactional integrity.
>
SNIP

> To me, if user details are in a directory, = detection of user
> preferences seems to be orthogonal to the OPES = transaction.
> *Identification* of the user within the OPES = transaction may
> be useful though.
>
> Rob
SNIP

-------- abbie

I agree, detection of user preferences is needed. I = also believe that Identification of the user within the OPES = transaction is more than useful, it may be needed.





------_=_NextPart_001_01C2D732.9FEF27E4-- From owner-ietf-openproxy@mail.imc.org Tue Feb 18 05:14:22 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA24043 for ; Tue, 18 Feb 2003 05:14:20 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1I9sMf26339 for ietf-openproxy-bks; Tue, 18 Feb 2003 01:54:22 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1I9sMd26335 for ; Tue, 18 Feb 2003 01:54:22 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1I9s8A18729; Tue, 18 Feb 2003 04:54:08 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C64V72>; Tue, 18 Feb 2003 04:54:09 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E74646@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: "The Purple Streak, Hilarie Orman" , batuner@attbi.com Cc: ietf-openproxy@imc.org Subject: RE: SMTP filtering use case Date: Tue, 18 Feb 2003 04:54:07 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D733.ADD4DC2C" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D733.ADD4DC2C Content-Type: text/plain Hi, see comments inline abbie > -----Original Message----- > From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu] > Sent: Monday, February 17, 2003 5:55 PM > To: batuner@attbi.com > Cc: ietf-openproxy@imc.org > Subject: RE: SMTP filtering use case > > > > On Mon, 17 Feb 2003 at 17:17:18 -0500 Oskar Batuner avowed, > in replying to Alex Rousskov: > > > > It looks like the big question here is about control > over user > > > preferences. To me, it seems unreasonable to assume that > all user > > > preferences are handled at the OPES processor. It is just > too rigid of > > > a model and often very inefficient since the processor > knows very > > > little about filtering and the filter (OPES server) may > need to know a > > > lot. If we restrict user preference handling to OPES > processor, we > > end > up passing a lot of preference information to the > OPES server > > just so > that it can process a transaction. > > > I'm afraid we are going too deep with analysis of this specific > > scenario. > > If I understand correctly Martin used preferences just as > an example of > > the reason why callout processor may produce multiple responses > > for a single request. > > Discussions that reveal schisms in the meaning of the > architecture are very important, whether or not it was the > intention of the original message. The lack of specifics to > date has been difficult for everyone, and I'm sure that there > are more surprises lurking as we discuss a real protocol for > real uses. > ---------- abbie agree, this is why we said that the fun begins now. > > There may be any number of reasons for the callout processor to > > produce multi-message response, the protocol just SHOULD > (MUST?) support > > multiple responses while preserving transactional integrity. > > > We should not (MUST NOT) impose any limitation on what > > information is available to the callout processor (including > > preferences) and how OPES processor and callout processor assign > > tasks between themselves while performing a collaborative > > processing - this is application specifics and should be > > out of scope for protocol design. > > I think it's not as collaborative and loose and all that. > The OPES processor is the enforcement point for privacy. > It's designed to be a very fast and fine-grained, > policy-based, application-layer switch. If it is not the > repository for user preferences, then all the policy > enforcement and authentication requirements get pushed to the > callout servers. This leaves the OPES processor with almost > no function - it might as well be a dumb hardware switch > operating at layers 4 through 7. > > Hilarie > ---------- abbie Hilarie, good points (agree with u here). We can not assume that the callout server will be the PEP (or at least the main one). Otherwise, we get in trobule if the callout server is in different administration domain, etc... The way I look at it, the OPES processor may use different callout servers based on different services, so this means that the OPES processor must be the PEP, the callout server can be a secondary PEP if chaining is used. abbie ------_=_NextPart_001_01C2D733.ADD4DC2C Content-Type: text/html RE: SMTP filtering use case

Hi,
see comments inline

abbie

> -----Original Message-----
> From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu]
> Sent: Monday, February 17, 2003 5:55 PM
> To: batuner@attbi.com
> Cc: ietf-openproxy@imc.org
> Subject: RE: SMTP filtering use case
>
>
>
> On Mon, 17 Feb 2003 at 17:17:18 -0500 Oskar Batuner avowed,
> in replying to Alex Rousskov:
>
> >  > It looks like the big question here is about control
> over user  >
> > preferences. To me, it seems unreasonable to assume that
> all user  >
> > preferences are handled at the OPES processor. It is just
> too rigid of 
> > > a model and often very inefficient since the processor
> knows very  >
> > little about filtering and the filter (OPES server) may
> need to know a 
> > > lot. If we restrict user preference handling to OPES
> processor, we
> > end  > up passing a lot of preference information to the
> OPES server
> > just so  > that it can process a transaction.
>
> >  I'm afraid we are going too deep with analysis of this specific
> > scenario.
> >  If I understand correctly Martin used preferences just as
> an example of
> >  the reason why callout processor may produce multiple responses
> >  for a single request.
>
> Discussions that reveal schisms in the meaning of the
> architecture are very important, whether or not it was the
> intention of the original message.  The lack of specifics to
> date has been difficult for everyone, and I'm sure that there
> are more surprises lurking as we discuss a real protocol for
> real uses.
>

---------- abbie
agree, this is why we said that the fun begins now.


> >  There may be any number of reasons for the callout processor to
> >  produce multi-message response, the protocol just SHOULD
> (MUST?) support
> >  multiple responses while preserving transactional integrity.
>
> >  We should not (MUST NOT) impose any limitation on what
> >  information is available to the callout processor (including
> >  preferences) and how OPES processor and callout processor assign
> >  tasks between themselves while performing a collaborative
> >  processing - this is application specifics and should be
> >  out of scope for protocol design.
>
> I think it's not as collaborative and loose and all that. 
> The OPES processor is the enforcement point for privacy. 
> It's designed to be a very fast and fine-grained,
> policy-based, application-layer switch.  If it is not the
> repository for user preferences, then all the policy
> enforcement and authentication requirements get pushed to the
> callout servers.  This leaves the OPES processor with almost
> no function - it might as well be a dumb hardware switch
> operating at layers 4 through 7.
>
> Hilarie
>

---------- abbie

Hilarie, good points (agree with u here).

We can not assume that the callout server will be the PEP (or at least the main one). Otherwise, we get in trobule if the callout server is in different administration domain, etc... The way I look at it, the OPES processor may use different callout servers based on different services, so this means that the OPES processor must be the PEP, the callout server can be a secondary PEP if chaining is used.

abbie












------_=_NextPart_001_01C2D733.ADD4DC2C-- From owner-ietf-openproxy@mail.imc.org Tue Feb 18 05:21:41 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA24171 for ; Tue, 18 Feb 2003 05:21:40 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1I9vhr26444 for ietf-openproxy-bks; Tue, 18 Feb 2003 01:57:43 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1I9vgd26440 for ; Tue, 18 Feb 2003 01:57:42 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1I9vQd15635; Tue, 18 Feb 2003 04:57:26 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C64V84>; Tue, 18 Feb 2003 04:57:27 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E74647@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , OPES Group Subject: RE: protocol core now, transport/encoding later Date: Tue, 18 Feb 2003 04:57:23 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D734.22903EA8" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D734.22903EA8 Content-Type: text/plain hi, see A. beck response regarding the requirement draft. on the otherhadn, I agree with you that the resulting protocol better meet the requirement draft (at least for most of the cases) abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Monday, February 17, 2003 3:40 PM > To: OPES Group > Subject: RE: protocol core now, transport/encoding later > > > > > On Mon, 17 Feb 2003, Abbie Barbir wrote: > > > -- good questions that u raise, however, the questions should have > > been answered in the protocol requirements draft, so i > suggest that we > > start from there. > > Abbie, > > My understanding is that the requirements draft does > not answer those questions. Moreover, there is no clear > boundary between protocol requirements and actual decision > points in protocol design. The requirements may be very > specific or quite general. We ended up with general > requirements. Fine. The requirements stage is now behind us, > and I would like to avoid going back unless absolutely > necessary. Regardless of how we answer the questions raised, > the resulting protocol is likely to satisfy the requirements > draft, and that's all we should care about. It is time for > specific protocol-level work, IMO. > > If my understanding is wrong, and some of the questions are > already clearly answered by the requirements draft, please > quote these answers, and we will move on. > > Thank you, > > Alex. > > -- > | HTTP performance - Web > Polygraph benchmark www.measurement-factory.com | HTTP > compliance+ - Co-Advisor test suite > | all of the above - PolyBox appliance > ------_=_NextPart_001_01C2D734.22903EA8 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: protocol core now, transport/encoding later

hi,

see A. beck response regarding the requirement = draft.

on the otherhadn, I agree with you that the resulting = protocol better meet the requirement draft (at least for most of the = cases)

abbie


> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Monday, February 17, 2003 3:40 PM
> To: OPES Group
> Subject: RE: protocol core now, = transport/encoding later
>
>
>
>
> On Mon, 17 Feb 2003, Abbie Barbir wrote:
>
> > -- good questions that u raise, however, = the questions should have
> > been answered in the protocol requirements = draft, so i
> suggest that we
> > start from there.
>
> Abbie,
>
>       My understanding = is that the requirements draft does
> not answer those questions. Moreover, there is = no clear
> boundary between protocol requirements and = actual decision
> points in protocol design. The requirements may = be very
> specific or quite general. We ended up with = general
> requirements. Fine. The requirements stage is = now behind us,
> and I would like to avoid going back unless = absolutely
> necessary. Regardless of how we answer the = questions raised,
> the resulting protocol is likely to satisfy the = requirements
> draft, and that's all we should care about. It = is time for
> specific protocol-level work, IMO.
>
> If my understanding is wrong, and some of the = questions are
> already clearly answered by the requirements = draft, please
> quote these answers, and we will move = on.
>
> Thank you,
>
> Alex.
>
> --
>          = ;            = ;       | HTTP performance - Web
> Polygraph benchmark www.measurement-factory.com = | HTTP
> compliance+ - Co-Advisor test suite
>          = ;            = ;       | all of the above - PolyBox = appliance
>

------_=_NextPart_001_01C2D734.22903EA8-- From owner-ietf-openproxy@mail.imc.org Tue Feb 18 06:07:30 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA25157 for ; Tue, 18 Feb 2003 06:07:27 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1IAdAJ27598 for ietf-openproxy-bks; Tue, 18 Feb 2003 02:39:10 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1IAd9d27594 for ; Tue, 18 Feb 2003 02:39:09 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1IAd1d23180; Tue, 18 Feb 2003 05:39:02 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C64WHR>; Tue, 18 Feb 2003 05:39:01 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E74649@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Martin Stecher Cc: ietf-openproxy@imc.org Subject: RE: Start on OPES protocol work Date: Tue, 18 Feb 2003 05:39:00 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D739.F29F10F6" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D739.F29F10F6 Content-Type: text/plain MArtin, for BEEP check, http://beepcore.org/beepcore/specsdocs.jsp http://www-106.ibm.com/developerworks/xml/library/x-beep2.html http://www-106.ibm.com/developerworks/xml/library/x-beep/index.html abbie > -----Original Message----- > From: Martin Stecher [mailto:martin.stecher@webwasher.com] > Sent: Monday, February 17, 2003 5:41 AM > To: The Purple Streak, Hilarie Orman > Cc: ietf-openproxy@imc.org > Subject: RE: Start on OPES protocol work > > > > Hi Hilarie, > > > > > 1. I believe that the protocol requirements stipulate the use of > > independent "channels", and those were put in exactly for > the the case > > you are discussing for preview. If channels don't solve > the problem, > > we need to know why. > > draft-ietf-opes-protocol-reqs-03 does not mention the > "channels", does it? Till this weekend I was not aware that > the "channels" concept already implies that each fragment of > an application message is sent in its own callout message via > the channel. Is this correct? If yes, I appreciate this > concept very much and I think it is exactly what I thought > about solving the out-of-sync and preview problems. > > > > > > 2. For the SMTP case, let's suppose that the site policy > > requires virus > > scanning, but some subscribers to the OPES box want > potential viruses > > removed, and some simply want them flagged. Assume a > message has been > > sent to several recipients of a domain with an OPES virus scanning > > service, and its body is denoted as "A" (identified by its hash, I > > suppose). If it's the OPES processor that holds the user > preferences, > > then it can handle the process of sending A to a callout server for > > virus scanning and asking for both removal and flagging, possibly in > > one request, but at worst in two. Ah, I see, you'd like to > eliminate > > the two requests, and have a request type that asks for multiple > > services and multiple replies. This is different from asking for > > multiple services and one reply. A notation for this might > > be something > > like: > > > > Request(data=D; service=A AND service=B) > > Reply(data=A(B(D))) > > > > Request(data=D; service=A; service=B) > > Reply(data=A(D); data=B(D)) > > > > Have I captured the requirement correctly? > > I am not sure. Please have a look into my posting "SMTP > filtering use case". > > > > > 3. I know the arguments for simple formats, but please give > something > > specific about your objection to BEEP. My objection to XML > encoding > > is that it requires many instructions to parse it into tag > pairs and > > parameters, to create a parse tree, and to traverse the tree. When > > comparing this to direct indexing, or even scans for end-of-header, > > end-of-line, XML is very slow. But, experience has shown > that there > > are distinct limits to what can be accomplished with fixed > parameters > > positions and bit encodings, and even TCP has had a great deal of > > trouble staying within its header limitations. Up at the > application > > layer, I think some compromise will be necessary. I'd thought BEEP > > was actually > > a good point to begin looking for that compromise point. What is so > > awful about it? > > > I am really sorry that I started to name protocols. I didn't > want to get into this discussion now. On the other hand it > shows that existing protocols have interesting concepts that > are very worth to be known and understood while discussing > OPES protocol needs and I doubt that I would really look in > the protocol's details without this discussion. > > Unfortunately I had only a brief look into RFC 3080 (BEEP > core) before sending my first message last week. And in there > I found all examples full of XML structure, always repeating > same Content-Type header with every single small message and > that little PASCAL reminding "END" literal. > > So on my first glance it did not look very efficient to me. > I thought that if each application message's fragment is > encapsulated in its own BEEP message this would be a big > overhead. Seems that this is not so. > > By learning a little more about it over the weekend and > starting to understand the "channels" concept, I agree that > BEEP is very worth to be considered. It is not "awful" (and I > never said so). > > But still we should not elect the one or other protocol now. > What we rather need is a common understanding about the best > concepts and how they could fulfill the requirements and > solve the questions and problems that we list. > > What other BEEP documents than RFC 3080 should be studied? > Is there a good summary about the concepts that could help > OPES? If not, could somebody compile one? > > > Martin > > > ------_=_NextPart_001_01C2D739.F29F10F6 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: Start on OPES protocol work

MArtin,

for BEEP check,

http://beepcore.org/beepcore/specsdocs.jsp

http://www-106.ibm.com/developerworks/xml/library/x-be= ep2.html

http://www-106.ibm.com/developerworks/xml/library/x-be= ep/index.html

abbie


> -----Original Message-----
> From: Martin Stecher [mailto:martin.stecher@webwa= sher.com]
> Sent: Monday, February 17, 2003 5:41 AM
> To: The Purple Streak, Hilarie Orman
> Cc: ietf-openproxy@imc.org
> Subject: RE: Start on OPES protocol work
>
>
>
> Hi Hilarie,
>
> >
> > 1. I believe that the protocol = requirements stipulate the use of
> > independent "channels", and = those were put in exactly for
> the the case
> > you are discussing for preview.  If = channels don't solve
> the problem,
> > we need to know why.
>
> draft-ietf-opes-protocol-reqs-03 does not = mention the
> "channels", does it? Till this = weekend I was not aware that
> the "channels" concept already = implies that each fragment of
> an application message is sent in its own = callout message via
> the channel. Is this correct? If yes, I = appreciate this
> concept very much and I think it is exactly = what I thought
> about solving the out-of-sync and preview = problems.
>
>
> >
> > 2. For the SMTP case, let's suppose that = the site policy
> > requires virus
> > scanning, but some subscribers to the OPES = box want
> potential viruses
> > removed, and some simply want them = flagged.  Assume a
> message has been
> > sent to several recipients of a domain = with an OPES virus scanning
> > service, and its body is denoted as = "A" (identified by its hash, I
> > suppose).  If it's the OPES processor = that holds the user
> preferences,
> > then it can handle the process of sending = A to a callout server for
> > virus scanning and asking for both removal = and flagging, possibly in
> > one request, but at worst in two.  = Ah, I see, you'd like to
> eliminate
> > the two requests, and have a request type = that asks for multiple
> > services and multiple replies.  This = is different from asking for
> > multiple services and one reply.  A = notation for this might
> > be something
> > like:
> >
> >   Request(data=3DD; service=3DA = AND service=3DB)
> >   Reply(data=3DA(B(D)))
> >
> >   Request(data=3DD; service=3DA; = service=3DB)
> >   Reply(data=3DA(D); = data=3DB(D))
> >
> > Have I captured the requirement = correctly?
>
> I am not sure. Please have a look into my = posting "SMTP
> filtering use case".
>
> >
> > 3. I know the arguments for simple = formats, but please give
> something
> > specific about your objection to = BEEP.  My objection to XML
> encoding
> > is that it requires many instructions to = parse it into tag
> pairs and
> > parameters, to create a parse tree, and to = traverse the tree.  When
> > comparing this to direct indexing, or even = scans for end-of-header,
> > end-of-line, XML is very slow.  But, = experience has shown
> that there
> > are distinct limits to what can be = accomplished with fixed
> parameters
> > positions and bit encodings, and even TCP = has had a great deal of
> > trouble staying within its header = limitations.  Up at the
> application
> > layer, I think some compromise will be = necessary.  I'd thought BEEP
> > was actually
> > a good point to begin looking for that = compromise point.  What is so
> > awful about it?
>
>
> I am really sorry that I started to name = protocols. I didn't
> want to get into this discussion now. On the = other hand it
> shows that existing protocols have interesting = concepts that
> are very worth to be known and understood while = discussing
> OPES protocol needs and I doubt that I would = really look in
> the protocol's details without this = discussion.
>
> Unfortunately I had only a brief look into RFC = 3080 (BEEP
> core) before sending my first message last = week. And in there
> I found all examples full of XML structure, = always repeating
> same Content-Type header with every single = small message and
> that little PASCAL reminding "END" = literal.
>
> So on my first glance it did not look very = efficient to me.
> I thought that if each application message's = fragment is
> encapsulated in its own BEEP message this would = be a big
> overhead. Seems that this is not so.
>
> By learning a little more about it over the = weekend and
> starting to understand the "channels" = concept, I agree that
> BEEP is very worth to be considered. It is not = "awful" (and I
> never said so).
>
> But still we should not elect the one or other = protocol now.
> What we rather need is a common understanding = about the best
> concepts and how they could fulfill the = requirements and
> solve the questions and problems that we = list.
>
> What other BEEP documents than RFC 3080 should = be studied?
> Is there a good summary about the concepts that = could help
> OPES? If not, could somebody compile = one?
>
>
> Martin
>
>
>

------_=_NextPart_001_01C2D739.F29F10F6-- From owner-ietf-openproxy@mail.imc.org Tue Feb 18 11:07:41 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA00879 for ; Tue, 18 Feb 2003 11:07:40 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1IFhAV05966 for ietf-openproxy-bks; Tue, 18 Feb 2003 07:43:10 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1IFh7d05962 for ; Tue, 18 Feb 2003 07:43:08 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id JTTJBP57; 18 Feb 2003 16:43:06 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: Start on OPES protocol work X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Tue, 18 Feb 2003 16:40:03 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C10E@hermes.webwasher.com> Thread-Topic: Start on OPES protocol work Thread-Index: AcLW4HY9zq4Z8q0fSuWpaG/2cqfx/gAfQIzg From: "Martin Stecher" To: "Andre Beck" Cc: Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1IFh9d05963 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit Hi Andre, I think that the channel concept meets all requirements from section 3.4. The general channel concept may even be exactly what we have in 3.4. There is one detail in the channel concept in BEEP that I find so important that I like to repeat it because I don't see that we have a real requirement for it in the draft: Due to the sequence number, channel number and message size all given in the header of each message, the concept strictly implies that every fragment of an application message is sent within its own message. Example: If a HTTP message is received in chunked encoding and the OPES processor wants to forward some first chunks while the rest did not yet arrive, it has to use a complete message which ends after this first HTTP message part. Later chunks that are received come within their own callout message. This way, a multiple preview concept and implementations that are not susceptible to the out-of-sync problem become easy. Do you share this view? Do you agree with me that this is an important feature we have to add to the OPES protocol? Regards Martin > -----Original Message----- > From: Andre Beck [mailto:abeck@bell-labs.com] > Sent: Tuesday, February 18, 2003 1:01 AM > To: Martin Stecher > Cc: ietf-openproxy@imc.org > Subject: Re: Start on OPES protocol work > > > Hi Martin, > > > draft-ietf-opes-protocol-reqs-03 does not mention the > "channels", does it? > > The -03 version of the draft refers to them as callout > connections (see > section 3.4 of the draft). I believe the callout connection > concept as > described in the draft is very similar to the channel concept in BEEP. > > -Andre > > From owner-ietf-openproxy@mail.imc.org Tue Feb 18 11:12:17 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA00980 for ; Tue, 18 Feb 2003 11:12:15 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1IFpCq06186 for ietf-openproxy-bks; Tue, 18 Feb 2003 07:51:12 -0800 (PST) Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1IFp9d06180 for ; Tue, 18 Feb 2003 07:51:09 -0800 (PST) Received: from oskar3 (h00a0c5e5337d.ne.client2.attbi.com[24.218.184.133]) by rwcrmhc51.attbi.com (rwcrmhc51) with SMTP id <2003021815510405100fjmpke>; Tue, 18 Feb 2003 15:51:04 +0000 From: "Oskar Batuner" To: "The Purple Streak, Hilarie Orman" Cc: Subject: RE: SMTP filtering use case Date: Tue, 18 Feb 2003 10:51:37 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal In-Reply-To: <200302172255.h1HMtCq32349@localhost.localdomain> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit Hilarie, I did not meant to say that this discussion has no point. I'm just trying to look at the protocol features that are needed to support the application requirements that surfaced in the discussion about Martin's scenario. Essentially this are transaction integrity and one-to-many mapping between the protocol messages and the application messages. Application architecture discussion in fact raises one more issue - performance requirements for OPES processor and callout server. One approach is to go from OPES server performance required to keep up with the request flow. If callout server has to be in this flow the callout protocol should be light weight, SOAP and XML do not look good for that. On another hand we may look at the task performed by the callout processor. If each request requires significant processing protocol parsing becomes comparatively less important - even for heavy weight protocols. If callout server is powerful enough to keep up with processing requirements it should be able to do parsing too (this is the situation when functional request processing is much more demanding than parsing of heavyweight protocol). Problem of keeping up with performance requirements that exceed capabilities of single server may be approached from two directions - either by building a server farm or by distributing requests from OPES processors to multiple callout servers. We may look at protocol features that are helpful to support each approach. Oskar > -----Original Message----- > From: The Purple Streak, Hilarie Orman [mailto:ho@alum.mit.edu] > Sent: Monday, February 17, 2003 5:55 PM > To: batuner@attbi.com > Cc: ietf-openproxy@imc.org > Subject: RE: SMTP filtering use case > > > On Mon, 17 Feb 2003 at 17:17:18 -0500 Oskar Batuner avowed, in replying > to Alex Rousskov: > > > > It looks like the big question here is about control over user > > > preferences. To me, it seems unreasonable to assume that all user > > > preferences are handled at the OPES processor. It is just too rigid of > > > a model and often very inefficient since the processor knows very > > > little about filtering and the filter (OPES server) may need to know a > > > lot. If we restrict user preference handling to OPES processor, we end > > > up passing a lot of preference information to the OPES server just so > > > that it can process a transaction. > > > I'm afraid we are going too deep with analysis of this specific > scenario. > > If I understand correctly Martin used preferences just as an example of > > the reason why callout processor may produce multiple responses > > for a single request. > > Discussions that reveal schisms in the meaning of the architecture are > very important, whether or not it was the intention of the original > message. The lack of specifics to date has been difficult for everyone, > and I'm sure that there are more surprises lurking as we discuss a real > protocol for real uses. > > > There may be any number of reasons for the callout processor to > > produce multi-message response, the protocol just SHOULD (MUST?) support > > multiple responses while preserving transactional integrity. > > > We should not (MUST NOT) impose any limitation on what > > information is available to the callout processor (including > > preferences) and how OPES processor and callout processor assign > > tasks between themselves while performing a collaborative > > processing - this is application specifics and should be > > out of scope for protocol design. > > I think it's not as collaborative and loose and all that. The OPES > processor > is the enforcement point for privacy. It's designed to be a very fast > and fine-grained, policy-based, application-layer switch. If it is not > the repository for user preferences, then all the policy enforcement > and authentication requirements get pushed to the callout servers. This > leaves the OPES processor with almost no function - it might as well be > a dumb hardware switch operating at layers 4 through 7. > > Hilarie > > From owner-ietf-openproxy@mail.imc.org Tue Feb 18 11:31:15 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA01267 for ; Tue, 18 Feb 2003 11:31:14 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1IG1is06398 for ietf-openproxy-bks; Tue, 18 Feb 2003 08:01:44 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1IG1hd06394 for ; Tue, 18 Feb 2003 08:01:43 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1IG1ieM096686 for ; Tue, 18 Feb 2003 09:01:44 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1IG1iHD096685; Tue, 18 Feb 2003 09:01:44 -0700 (MST) (envelope-from rousskov) Date: Tue, 18 Feb 2003 09:01:44 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: Start on OPES protocol work In-Reply-To: <87609AFB433BD5118D5E0002A52CD75404E74649@zcard0k6.ca.nortel.com> Message-ID: References: <87609AFB433BD5118D5E0002A52CD75404E74649@zcard0k6.ca.nortel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Tue, 18 Feb 2003, Abbie Barbir wrote: > http://beepcore.org/beepcore/specsdocs.jsp > http://www-106.ibm.com/developerworks/xml/library/x-beep2.html > http://www-106.ibm.com/developerworks/xml/library/x-beep/index.html My understanding is that supporting BEEP requires supporting XML because , , , and other XML elements have to be supported. Please correct me if I am wrong! I just want to make sure that XML-enemies know that BEEP relies on XML... It is possible to limit XML support to rigid parsing of the few documented elements (we have done that in Web Polygraph benchmark, for example). However, such limited support is the "wrong thing to do" in general and is likely to cause compatibility problems in the future. Alex. From owner-ietf-openproxy@mail.imc.org Tue Feb 18 11:44:25 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA01652 for ; Tue, 18 Feb 2003 11:44:24 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1IGENr06820 for ietf-openproxy-bks; Tue, 18 Feb 2003 08:14:23 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1IGEMd06815 for ; Tue, 18 Feb 2003 08:14:22 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1IGENeM096932 for ; Tue, 18 Feb 2003 09:14:23 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1IGENMN096931; Tue, 18 Feb 2003 09:14:23 -0700 (MST) (envelope-from rousskov) Date: Tue, 18 Feb 2003 09:14:23 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: SMTP filtering use case In-Reply-To: <87609AFB433BD5118D5E0002A52CD75404E74646@zcard0k6.ca.nortel.com> Message-ID: References: <87609AFB433BD5118D5E0002A52CD75404E74646@zcard0k6.ca.nortel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Mon, 17 Feb 2003, The Purple Streak, Hilarie Orman wrote: > I think it's not as collaborative and loose and all that. The OPES > processor is the enforcement point for privacy. It's designed to be > a very fast and fine-grained, policy-based, application-layer > switch. If it is not the repository for user preferences, then all > the policy enforcement and authentication requirements get pushed to > the callout servers. This leaves the OPES processor with almost no > function - it might as well be a dumb hardware switch operating at > layers 4 through 7. On Tue, 18 Feb 2003, Abbie Barbir wrote: > We can not assume that the callout server will be the PEP (or at > least the main one). Otherwise, we get in trobule if the callout > server is in different administration domain, etc... The way I look > at it, the OPES processor may use different callout servers based on > different services, so this means that the OPES processor must be > the PEP, the callout server can be a secondary PEP if chaining is > used. I would suggest looking at the privacy/policy enforcement from the content producer and content consumer points of view instead of from OPES point of view. OPES must satisfy producer and consumer expectations and must address IAB concerns. This does not imply that we have to place enforcement in one place: OPES processor or OPES server. It simply means that the policies must be enforced. The more flexible the protocol is, the better. Neither the producer nor the consumer care where/how exactly their policies are enforced as long as the policies are enforced. In some deployments, the OPES processor may be a dumb hardware switch operating at L7. Fine! Some scenarios will make OPES server nearly as dumb and place policy enforcement on the OPES processor. Great. If we can permit both scenarios while enforcing the policies and addressing concerns from producer and consumer points of view, we should do that. At this time, it is too early to tell whether we can so let's not assume any restrictions on policy enforcer location until we have to introduce them. Thanks, Alex. From owner-ietf-openproxy@mail.imc.org Tue Feb 18 12:07:09 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA02083 for ; Tue, 18 Feb 2003 12:07:08 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1IGaaq07427 for ietf-openproxy-bks; Tue, 18 Feb 2003 08:36:36 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1IGaYd07423 for ; Tue, 18 Feb 2003 08:36:35 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1IGaLd05160; Tue, 18 Feb 2003 11:36:21 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C640R1>; Tue, 18 Feb 2003 11:36:22 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404E749D4@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , ietf-openproxy@imc.org Cc: Marshall Rose Subject: RE: Start on OPES protocol work Date: Tue, 18 Feb 2003 11:36:12 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D76B.D8EBB556" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D76B.D8EBB556 Content-Type: text/plain Alex, I think that u r right with your assumption. Marshall, can u elaborate. abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Tuesday, February 18, 2003 11:02 AM > To: ietf-openproxy@imc.org > Subject: RE: Start on OPES protocol work > > > > On Tue, 18 Feb 2003, Abbie Barbir wrote: > > > http://beepcore.org/beepcore/specsdocs.jsp > > http://www-106.ibm.com/developerworks/xml/library/x-beep2.html > > http://www-106.ibm.com/developerworks/xml/library/x-beep/index.html > > My understanding is that supporting BEEP requires supporting > XML because , , , and other XML > elements have to be supported. Please correct me if I am > wrong! I just want to make sure that XML-enemies know that > BEEP relies on XML... > > It is possible to limit XML support to rigid parsing of the > few documented elements (we have done that in Web Polygraph > benchmark, for example). However, such limited support is the > "wrong thing to do" in general and is likely to cause > compatibility problems in the future. > > Alex. > ------_=_NextPart_001_01C2D76B.D8EBB556 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: Start on OPES protocol work

Alex,
I think that u r right with your assumption.
Marshall, can u elaborate.

abbie


> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Tuesday, February 18, 2003 11:02 = AM
> To: ietf-openproxy@imc.org
> Subject: RE: Start on OPES protocol work
>
>
>
> On Tue, 18 Feb 2003, Abbie Barbir wrote:
>
> > http://beepcore.org/beepcore/specsdocs.jsp
> > http://www-106.ibm.com/developerworks/xml/library/x-be= ep2.html
> > http://www-106.ibm.com/developerworks/xml/library/x-be= ep/index.html
>
> My understanding is that supporting BEEP = requires supporting
> XML because <start>, <greeting>, = <profile>, and other XML
> elements have to be supported. Please correct = me if I am
> wrong! I just want to make sure that = XML-enemies know that
> BEEP relies on XML...
>
> It is possible to limit XML support to rigid = parsing of the
> few documented elements (we have done that in = Web Polygraph
> benchmark, for example). However, such limited = support is the
> "wrong thing to do" in general and is = likely to cause
> compatibility problems in the future.
>
> Alex.
>

------_=_NextPart_001_01C2D76B.D8EBB556-- From owner-ietf-openproxy@mail.imc.org Tue Feb 18 12:13:13 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA02180 for ; Tue, 18 Feb 2003 12:13:11 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1IGo7p07868 for ietf-openproxy-bks; Tue, 18 Feb 2003 08:50:07 -0800 (PST) Received: from shego.dbc.mtview.ca.us ([65.125.189.56]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1IGo6d07863 for ; Tue, 18 Feb 2003 08:50:06 -0800 (PST) Received: from shego.dbc.mtview.ca.us (localhost [127.0.0.1]) by shego.dbc.mtview.ca.us (8.12.6/8.12.6) with SMTP id h1IGo1Kr000320; Tue, 18 Feb 2003 08:50:01 -0800 (PST) Date: Tue, 18 Feb 2003 08:50:00 -0800 From: Marshall Rose To: "Abbie Barbir" Cc: rousskov@measurement-factory.com, ietf-openproxy@imc.org Subject: Re: Start on OPES protocol work Message-Id: <20030218085000.4ae53f67.mrose@dbc.mtview.ca.us> In-Reply-To: <87609AFB433BD5118D5E0002A52CD75404E749D4@zcard0k6.ca.nortel.com> References: <87609AFB433BD5118D5E0002A52CD75404E749D4@zcard0k6.ca.nortel.com> Organization: Dover Beach Consulting, Inc. X-Mailer: Sylpheed version 0.8.8claws (GTK+ 1.2.10; i386--netbsdelf) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit > > > http://beepcore.org/beepcore/specsdocs.jsp > > > http://www-106.ibm.com/developerworks/xml/library/x-beep2.html > > > http://www-106.ibm.com/developerworks/xml/library/x-beep/index.html > > > > My understanding is that supporting BEEP requires supporting > > XML because , , , and other XML > > elements have to be supported. Please correct me if I am > > wrong! I just want to make sure that XML-enemies know that > > BEEP relies on XML... beep's channel management uses xml to format messages. similarly, you invoke sasl and tls using xml-formated messages. typically, these messages get sent once per session, to "tune" the connection the way you want it. besides that, you're perfectly free to format your own messages however you want, i.e., binary, 822, s-expressions, etc. > > It is possible to limit XML support to rigid parsing of the > > few documented elements (we have done that in Web Polygraph > > benchmark, for example). However, such limited support is the > > "wrong thing to do" in general and is likely to cause > > compatibility problems in the future. the xml used by beep for the channel management/sasl/tls stuff is purposefully restricted to permit minimal implementations. i've seen some implementations get by using printf and sscanf. /mtr From owner-ietf-openproxy@mail.imc.org Tue Feb 18 14:25:14 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA05813 for ; Tue, 18 Feb 2003 14:25:13 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1IJ0IC11311 for ietf-openproxy-bks; Tue, 18 Feb 2003 11:00:18 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1IJ0Hd11307 for ; Tue, 18 Feb 2003 11:00:17 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1IJWZnT012594; Tue, 18 Feb 2003 12:32:36 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1IJ11v24347; Tue, 18 Feb 2003 12:01:01 -0700 Date: Tue, 18 Feb 2003 12:01:01 -0700 Message-Id: <200302181901.h1IJ11v24347@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: rousskov@measurement-factory.com Cc: ietf-openproxy@imc.org In-reply-to: Yourmessage Subject: RE: SMTP filtering use case Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: It's very difficult to satisfy the IAB concerns by saying that callout servers will handle all policy - we aren't specifying the callout server applications. I think that scenarios which can be satisfied by a switch are just fine, but if they can be deployed today without any new protocols, then they aren't topics for discussion in this WG, except as negative examples, i.e., "you don't need OPES for that". Hilarie From owner-ietf-openproxy@mail.imc.org Tue Feb 18 14:40:36 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA06349 for ; Tue, 18 Feb 2003 14:40:34 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1IJJRI11902 for ietf-openproxy-bks; Tue, 18 Feb 2003 11:19:27 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1IJJPd11898 for ; Tue, 18 Feb 2003 11:19:25 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1IJJNeM002267; Tue, 18 Feb 2003 12:19:23 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1IJJNVW002266; Tue, 18 Feb 2003 12:19:23 -0700 (MST) (envelope-from rousskov) Date: Tue, 18 Feb 2003 12:19:23 -0700 (MST) From: Alex Rousskov To: "The Purple Streak, Hilarie Orman" cc: ietf-openproxy@imc.org Subject: RE: SMTP filtering use case In-Reply-To: <200302181901.h1IJ11v24347@localhost.localdomain> Message-ID: References: <200302181901.h1IJ11v24347@localhost.localdomain> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Tue, 18 Feb 2003, The Purple Streak, Hilarie Orman wrote: > It's very difficult to satisfy the IAB concerns by saying that > callout servers will handle all policy - we aren't specifying the > callout server applications. We are specifying the protocol the OPES server MUST implement to be called an OPES server. Our protocol may specify requirements that can only be satisfied if OPES processor and OPES server work together and do not violate each-other assumptions. That approach can both satisfy the IAB concerns and allow for different implementations/integrations of processors and servers to exist. In other words, ideally, we should only care about the visible end-to-end effect (the black-box approach) of a compliant processor+server combo, not about what a particular OPES processor or server may end up doing to satisfy our MUSTs. > I think that scenarios which can be satisfied by a switch are just > fine, but if they can be deployed today without any new protocols, > then they aren't topics for discussion in this WG, except as > negative examples, i.e., "you don't need OPES for that". If they are deployed today without OPES, they will violate IAB concerns/requirements. That is why we are chartered, in part, to produce a new protocol that addresses IAB concerns. Alex. From owner-ietf-openproxy@mail.imc.org Tue Feb 18 18:47:30 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA11801 for ; Tue, 18 Feb 2003 18:47:27 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1INPMv19634 for ietf-openproxy-bks; Tue, 18 Feb 2003 15:25:22 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1INPKd19628 for ; Tue, 18 Feb 2003 15:25:20 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1INPNeM007957 for ; Tue, 18 Feb 2003 16:25:23 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1INPN0q007956; Tue, 18 Feb 2003 16:25:23 -0700 (MST) (envelope-from rousskov) Date: Tue, 18 Feb 2003 16:25:23 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: OPES protocol, pre-draft Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: To bootstrap creation of OPES protocol specs and to show one possible way to answer the 5 design questions raised on this list, I would like to propose the following protocol outline. If you think this is too premature to talk about protocol specifics, please suggest what must be done first. I believe this protocol core can be implemented on top of HTTP or BEEP. I am less sure about SOAP; I am not a SOAP expert. I think that the relaxed flow of the proposed protocol is a good idea, given the combination of robustness requirements and requirements to support a diverse set of application protocols. Mandatory ACKs and explicit client-server flow do little good when you are managing several semi-independent I/O buffers and have to handle, for example, OPES server disappearance/overload and multiple recipients of a single message. This relaxed flow and focus on data manipulation are the key design elements that make the protocol simple yet efficient and robust (I hope). The protocol is optimized for the common case when most data is forwarded "as is", unaffected by the OPES server. Removing this optimization can simplify the protocol further. -------------------- Contents: 1. Terminology and environment 2. OPES processor-server communication 2.1 Transaction-independent messages 2.2 Transaction-dependent messages 2.2.1 Common transaction-dependent properties 2.2.2 Major message types 3. Examples 4. Transport connections 5. Synchronization and error handling 1. Terminology and environment draft-ietf-opes-protocol-reqs-03.txt defines the following information flow: data provider --(original application message)--> -- [ OPES magic ] --> --(produced application messages)--> data consumers The original and "produced" (forwarded) messages together form an application protocol transaction. Note that there may be more than one produced application message resulting from a single original message. When application protocol transaction involves a request-response sequence (e.g., HTTP), the above scheme remains the same. There are just two related transactions now: provider1 --(client req)--> [ OPES ] --(proxy req)--> consumer1 provider2 --(server resp)-> [ OPES ] --(proxy resp)-> consumer2 Usually, provider1 is consumer2, and consumer1 is provider2. Multiple related transactions do not change the nature of the OPES protocol. The only difference is that more information may need to be passed from OPES processor to OPES server. For example, processing of the response flow may need some knowledge about the request flow. Individual transactions may become related. There may be no server response. Depending on the application protocol, there may be multiple server and/or proxy responses. 2. OPES processor-server communication OPES processor and OPES server exchange messages. The exchange is bidirectional. There is no clear client or server role. There is no clear request/response message category either. There are two major kinds of messages: transaction-independent and transaction-dependent. 2.1 Transaction-independent messages Transaction-independent messages exchange information unrelated to any specific application protocol transaction. They are used to enquire about supported OPES protocol options, control remote logging, exchange user preferences, check connectivity status, etc. Transaction-independent messages are not documented here. 2.2 Transaction-dependent messages Transaction-dependent messages are the core of the OPES protocol. These messages alter, block, redirect, clone application messages to be forwarded to data consumers. Each transaction-dependent message is associated with a single application transaction by means of a unique transaction identifier (xid). It may be important to keep in mind that transaction-dependent messages manipulate the state of these four buffers/connections and associated meta-information: - data producer (incoming) buffer at the OPES processor - data producer (incoming) buffer at the OPES server - data consumer (outgoing) buffer at the OPES server - data consumer (outgoing) buffer at the OPES processor The design keeps buffers as allows to prevent buffer overflows and discard buffered content when possible. Note that we rely on transport protocol to be both reliable and to stop sending us more data (eventually) if we stop reading it. TCP has both properties. 2.2.1 Common transaction-dependent properties Some transaction-dependent OPES messages share the following common properties. xid -- Application transaction identifier (Xaction ID) source -- Information about the data provider (i.e., the source of the application message). For messages originated from the OPES processor, the source describes the original data provider. For messages originated from the OPES server, the source describes what provider information should be presented to the data consumer; OPES server may need to change how the original information looks to the other application side. destinations -- One or more destinations. A single destination is information about the data consumer (i.e, the destination of the application message). For messages originated from the OPES processor, destination describes the consumer as intended by the producer. For messages originated from the OPES server, the destination is the data consumer that should be used by the OPES processor; OPES server may need to change the intended recipient. Depending on the application, OPES processor may need to check that all destinations have been covered by OPES server. data size -- Specific data size in octets OR a special token meaning "all" or "maximum". The all-token may only be used when requesting data, never when sending it. reason -- This should probably be a numeric status code with an optional information string. We will use just strings for now. 2.2.2 Major message types OPES processor may send the following transaction-dependent messages to the OPES server. [ Note: The XML-like syntax does NOT imply that the messages should be implemented using XML. Text or binary encodings can be used; the encoding decision is out of scope of this document. ] Informs OPES server about a new application transaction. This message should probably identify OPES service(s) requested for this transaction and other transaction-global info unrelated to data buffering, sources, or destinations. Informs OPES server about a new request from the data producer. Buffer IDentifier (bid) uniquely identifies the data buffer used for this request. Bid can probably be set to xid unless we expect to handle protocols that may merge requests before forwarding them. Sends [a portion of] application message from the data producer buffer to the OPES server. If "copied" flag is set, the OPES server may assume that the corresponding data is buffered at the processor and may refer to it using messages described below. Notifies OPES server that there will be no more data for this transaction (coming from the OPES processor) UNLESS OPES server explicitly asks for it using message described below Notifies OPES server that there will be no more data for this transaction (coming from the OPES processor) Notifies OPES server that there will be no more messages for this bid (coming from the OPES processor) Notifies OPES server that there will be no more messages for this transaction (coming from the OPES processor) OPES server may send the following transaction-dependent messages to the OPES processor. Informs OPES processor that OPES server may want to send data from source to destination(s). The Buffer IDentifier (bid) is unique for the (xid, source, destinations) triplet and identifies consumer buffer at the OPES server. There may be other buffers/messages (bid triplets) associated with the same transaction (xid). Xid comes from the corresponding xaction-start message send by the OPES processor. Tells OPES processor to send the attached data to the data consumer Tells OPES processor to use processor's own copy of the specified data to send to the data consumer. This message can only specify data fragments previously marked with "copied" flag in a message from OPES processor. Tells OPES processor that the server will never send data-asis message for the specified data range. This message can only specify data fragments previously marked with the "copied" flag in a message from OPES processor. This optional message may help OPES processor to free its resources. Tells OPES processor to send the specified data segment to the OPES server (probably in response to data-pause message from the OPES server). Notifies OPES processor that it should not send more data for this transaction until OPES server explicitly asks for it using data-need message described above Tells OPES processor that there will be no more data for this bid (coming from the OPES server) Notifies OPES server that there will be no more messages for this bid (coming from the OPES server) Notifies OPES server that there will be no more messages for this transaction (coming from the OPES server) Note: There needs to be a way for OPES server to tell OPES processor to terminate (or short-circuit) the forwarding of a message. This feature needs to be added to the protocol, but it should not change the overall design. 3. Examples Here is an example of (not) filtering an HTTP message based on HTTP headers: processor: processor: processor: processor: server: server: server: Note that xaction-end implies consumer-end implies data-end, and there is no reason for OPES processor to send a xaction-end message to server if the server already sent xaction-end message. The lines above are grouped about possible network I/O boundaries; thus, only two network data packets may be required to process a message if the OPES server decides it does not care based on the headers. Here is an example of redirecting an HTTP request by changing its destination info and corresponding HTTP headers: processor: processor: processor: processor: server: server: server: Finally, here is an example of modifying the "middle" part of HTTP message body. The OPES server switches the message encoding to chunked, to avoid buffering data to figure out new Content-Length. processor: processor: processor: processor: server: server: server: server: processor: server: processor: /* send modified chunk, tell processor to ignore the original */ server: server: processor: processor: server: server: Note that once the flow starts, there are no explicit synchronization points or waiting. The above message order is not the only one possible: most messages from the processor are not synchronized with most messages from the server. 4. Transport connections Transport connections would depend on the transport protocol (HTTP, BEEP, etc.). It is important to note that regardless of the transport protocol chosen, it is possible to multiplex messages from the OPES processor (or from the server) over several persistent connections -- transaction-dependent messages do not depend on "connection" properties except for the basic requirement that ordered messages use the same connection, in the right order. Simple connection-specific messages can be introduced if we want to support keep-alive checks or if we want to retry aborted connections. 5. Synchronization and error handling. The protocol has very few explicit dependencies between messages. It is trivial to imagine a case where incorrect processor or server implementation would result in deadlocks or other bad states. All sorts of deadlocks are resolved using timeouts. If there is no progress with the transaction for an admin-configurable time, the transaction is aborted. Aborting at OPES server side is easy: server: On the processor side, specific actions would depend on the protocol and state. For example, if no response bytes have been sent to an HTTP client yet, then an error response can be sent. It would be also possible, in some states, to eliminate OPES server from processing if it fails. Supporting this behavior would require having a copy of entire application messages even is OPES server tells us it does not need a copy. The exact behavior must be admin-configurable. --------------------------- This is just a start, of course. Many details are not specified yet. For example, can OPES server request that a sub-fragment of copied fragment is forwarded "as is"? What is the best way to handle modifications of a non-chunked HTTP transfer that change total message-length? Also note that many command/option names should probably be changed/polished. It is quite possible, for example, to make the protocol look like the next generation of ICAP if we want to. I believe the proposed protocol covers current major ICAP capabilities and supports a few desired optimizations mentioned on the mailing list. Is there anything major that the protocol core does not support and that needs to be supported? (besides security and authentication features that are irrelevant at this point). For example, should we add commands to control persistency of application-protocol connections from the OPES server (I do not think so)? Should we make it possible for OPES server to change the application protocol (I do not think so)? Thank you, Alex. -- | HTTP performance - Web Polygraph benchmark www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite | all of the above - PolyBox appliance From owner-ietf-openproxy@mail.imc.org Tue Feb 18 19:23:12 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA12503 for ; Tue, 18 Feb 2003 19:23:09 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1J01Pe20978 for ietf-openproxy-bks; Tue, 18 Feb 2003 16:01:25 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1J01Od20974 for ; Tue, 18 Feb 2003 16:01:25 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1J01LA18700 for ; Tue, 18 Feb 2003 19:01:21 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6VJKB>; Tue, 18 Feb 2003 19:01:22 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404EC8D47@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: ietf-openproxy@imc.org Subject: RE: SMTP filtering use case Date: Tue, 18 Feb 2003 19:01:22 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D7AA.0954A6B0" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D7AA.0954A6B0 Content-Type: text/plain Well, it is getting interesting. I would like each one of us (all Protocol participant)to state what we have agreed on so far from their point of view (from the various threads, it is hard to figure that out). I would like to make a summary of yes and no and maybe's for the protocol that could be (after further enhancements) presented in SFO. abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Tuesday, February 18, 2003 2:19 PM > To: The Purple Streak, Hilarie Orman > Cc: ietf-openproxy@imc.org > Subject: RE: SMTP filtering use case > > > > On Tue, 18 Feb 2003, The Purple Streak, Hilarie Orman wrote: > > > It's very difficult to satisfy the IAB concerns by saying > that callout > > servers will handle all policy - we aren't specifying the callout > > server applications. > > We are specifying the protocol the OPES server MUST implement > to be called an OPES server. Our protocol may specify > requirements that can only be satisfied if OPES processor and > OPES server work together and do not violate each-other > assumptions. That approach can both satisfy the IAB concerns > and allow for different implementations/integrations of > processors and servers to exist. > > In other words, ideally, we should only care about the > visible end-to-end effect (the black-box approach) of a compliant > processor+server combo, not about what a particular OPES processor or > server may end up doing to satisfy our MUSTs. > > > I think that scenarios which can be satisfied by a switch are just > > fine, but if they can be deployed today without any new protocols, > > then they aren't topics for discussion in this WG, except > as negative > > examples, i.e., "you don't need OPES for that". > > If they are deployed today without OPES, they will violate > IAB concerns/requirements. That is why we are chartered, in > part, to produce a new protocol that addresses IAB concerns. > > Alex. > > ------_=_NextPart_001_01C2D7AA.0954A6B0 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: SMTP filtering use case

Well,

it is getting interesting.

I would like each one of us (all Protocol = participant)to state what we have agreed on so far from their point of = view (from the various threads, it is hard to figure that out). =

I would like to make a summary of yes and no and = maybe's for the protocol that could be (after further enhancements) = presented in SFO. 

abbie


> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Tuesday, February 18, 2003 2:19 PM
> To: The Purple Streak, Hilarie Orman
> Cc: ietf-openproxy@imc.org
> Subject: RE: SMTP filtering use case
>
>
>
> On Tue, 18 Feb 2003, The Purple Streak, Hilarie = Orman wrote:
>
> > It's very difficult to satisfy the IAB = concerns by saying
> that callout
> > servers will handle all policy - we aren't = specifying the callout
> > server applications.
>
> We are specifying the protocol the OPES server = MUST implement
> to be called an OPES server. Our protocol may = specify
> requirements that can only be satisfied if OPES = processor and
> OPES server work together and do not violate = each-other
> assumptions. That approach can both satisfy the = IAB concerns
> and allow for different = implementations/integrations of
> processors and servers to exist.
>
> In other words, ideally, we should only care = about the
> visible end-to-end effect (the black-box = approach) of a compliant
> processor+server combo, not about what a = particular OPES processor or
> server may end up doing to satisfy our = MUSTs.
>
> > I think that scenarios which can be = satisfied by a switch are just
> > fine, but if they can be deployed today = without any new protocols,
> > then they aren't topics for discussion in = this WG, except
> as negative
> > examples, i.e., "you don't need OPES = for that".
>
> If they are deployed today without OPES, they = will violate
> IAB concerns/requirements. That is why we are = chartered, in
> part, to produce a new protocol that addresses = IAB concerns.
>
> Alex.
>
>

------_=_NextPart_001_01C2D7AA.0954A6B0-- From owner-ietf-openproxy@mail.imc.org Tue Feb 18 21:50:58 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA15924 for ; Tue, 18 Feb 2003 21:50:56 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1J2TXp24872 for ietf-openproxy-bks; Tue, 18 Feb 2003 18:29:33 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1J2TWd24868 for ; Tue, 18 Feb 2003 18:29:32 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1J31vnT026600; Tue, 18 Feb 2003 20:01:57 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1J2UFn13195; Tue, 18 Feb 2003 19:30:15 -0700 Date: Tue, 18 Feb 2003 19:30:15 -0700 Message-Id: <200302190230.h1J2UFn13195@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: rousskov@measurement-factory.com Cc: ietf-openproxy@imc.org In-reply-to: Yourmessage Subject: Re: OPES protocol, pre-draft Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: It's a good start. The OPES processor should make the decisions about what to send to the consumer. This might just be a matter of terminology, but the processor is in control of the source and destination and should not send messages to new destinations based on OPES server demands. The start message should identify the total length of the data, if it is available. This might stretch over several "bids", see below. The first response from the server should identify the new total length, if it is available. If the length will change, but the new size is unknown, the server should indicate this. There is some confusion about "destination" - the OPES server should never change the destination (i.e., the endpoint), so I don't see why it is needed. In the redirection example, it would be sufficient to change the headers, and the purpose of "destination" is a mystery to me. The relationship between the application-layer framing, the bid and offset, and the OPES framing is not clear from your examples. The application data may be transmitted to the OPES server a packet at a time - this will mean a different bid on every data message, if you literally mean that a bid is a buffer id. Otherwise, it should have some other name. Also, the information about a bid should include its total length. It should be possible to send the start and end messages on a separate transport connection for handling errors or congestion. The case discussed for multiple services, multiple responses, should be included. To support it, one needs multiple service lists and an id for each response. It should be possible to indicate that the transmitted data comes from several places in the bid. This allows the OPES processor to omit huge cookies and other junk; the response, by including this information, helps the process limit the state and parsing. Hilarie From owner-ietf-openproxy@mail.imc.org Wed Feb 19 01:18:15 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA19964 for ; Wed, 19 Feb 2003 01:18:13 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1J5uhE29368 for ietf-openproxy-bks; Tue, 18 Feb 2003 21:56:43 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1J5ugd29364 for ; Tue, 18 Feb 2003 21:56:42 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1J5ukeM016974 for ; Tue, 18 Feb 2003 22:56:46 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1J5ukEQ016973; Tue, 18 Feb 2003 22:56:46 -0700 (MST) (envelope-from rousskov) Date: Tue, 18 Feb 2003 22:56:46 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: Re: OPES protocol, pre-draft In-Reply-To: <200302190230.h1J2UFn13195@localhost.localdomain> Message-ID: References: <200302190230.h1J2UFn13195@localhost.localdomain> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Tue, 18 Feb 2003, The Purple Streak, Hilarie Orman wrote: > The OPES processor should make the decisions about what to send to > the consumer. This might just be a matter of terminology, but the > processor is in control of the source and destination and should not > send messages to new destinations based on OPES server demands. It looks like a design decision to me (i.e., it can be done both ways). I think that you may want destination modification by OPES servers if you want to satisfy Martin's requirement to be able to produce multiple SMTP messages (to several small groups of recipients) from one original SMTP message (to one large group of recipients). My understanding is that it MAY be OPES server responsibility to "split" the original destination address(es) into two groups. That is why I let OPES server to modify original destination info. Other destination-modification examples include request redirection (within a CDN or at the surrogate). Source-modification examples include anonymizing proxies. If there is a consensus that OPES server cannot modify source and destination info, then we can simplify the protocol a little bit. Is there a consensus regarding this design decision? Perhaps Abbie's poll will show... > The start message should identify the total length of the data, if it > is available. This might stretch over several "bids", see below. Good point. This extra info about anticipated message length should not hurt and may be used for resource pre-allocation purposes. It MUST NOT be treated as normative/final, of course. I will modify the messages accordingly. > The first response from the server should identify the new total length, > if it is available. If the length will change, but the new size is > unknown, the server should indicate this. Agreed, except we also need to support the case where the OPES server does not know whether the length will change. For example, if the server replaces "foo" with "blah" and "bleh" with "bar", it can tell the final length (and whether it will change) only after seeing all message content. To make things more general and symmetric, I would make it possible (but optional) to supply this non-normative length estimate with every relevant message, in both directions. > There is some confusion about "destination" - the OPES server should > never change the destination (i.e., the endpoint), so I don't see why > it is needed. In the redirection example, it would be sufficient to > change the headers, and the purpose of "destination" is a mystery to > me. See above for motivation. The destination is needed because the OPES processor needs to know where to connect to forward the request. It is Bad Design to have OPES processor guess that information from [possibly modified] message headers. This, again, assumes that we want OPES server to be able to modify destination addresses. If we do not want that, there is no need for OPES server to pass that info back to processor, of course. Note that source and destination information is meta-level information that is often not completely available from HTTP headers. Take interception and WCCP-controlled proxies for example. These intermediaries often have to get destination address based on IP-level details, not from HTTP headers. Similarly, the source information is usually not available in the request headers but may be required to route and modify the message. Moreover, the protocol should make it possible to exchange other meta-level information. For example, the time of the request may be important ("no porn surfing before 6pm!"). > The relationship between the application-layer framing, the bid and > offset, and the OPES framing is not clear from your examples. The > application data may be transmitted to the OPES server a packet at a > time - this will mean a different bid on every data message, if you > literally mean that a bid is a buffer id. Otherwise, it should have > some other name. Yes, a terminology/naming problem: By "buffer" in Buffer ID you probably mean "piece of memory that holds a data packet". I meant "logical structure that holds all data associated with the application connection/message". That is, my-buffer may consist of many your-buffers. Perhaps "buffer" should be replaced with "connection"? But "connection" is bad because OPES server does not really manage application connections. "Message" seems too overloaded? "Application message" (amid)? Will change bid to amid unless there are better ideas. This is just terminology though. "Bid/amid" is permanent for the single application message (original or produced). This ID should be used by processor and server to manage appropriate data structures associated with the corresponding application connection. > Also, the information about a bid should include its total length. Not sure why that would be needed. Moreover, the "total length" of the connection buffers (which is what bid identifies) may change at any time. OPES server should not care how buffering is done at the OPES processor side and vice versa. Perhaps your suggestion is a result of my poor choice of the word "buffer"? See above. > It should be possible to send the start and end messages on a > separate transport connection for handling errors or congestion. Yes, and it is possible. The start message is the first message for an "amid" so it can go on any connection (brand new or idle persistent). The end messages, if they indicate an immediate abort, do not have to be in order with data messages and, hence, can be sent on any OPES connection as well. Recall that there is no protocol-mandated relation between OPES connections and application transactions/connections. If one wants to sent something "out of order", they can (and face the consequences). > The case discussed for multiple services, multiple responses, should > be included. To support it, one needs multiple service lists and an > id for each response. I believe this is already supported, kind of. The "services" attribute of the transaction start message can have a list of services. The OPES server can initiate multiple consumer-start messages based on that list. Each consumer-start message from OPES server has a unique bid (amid). What is not clear to me is how the OPES server would know whether the services list is an OR, AND, or XOR, or something else. I suspect we need to support If-header logic from ICAP if we want to go down this route. I will polish the protocol once I understand the exact requirement here. Do we need to support some kind of service-to-response matching language here? I do not recall a clear answer in available OPES IDs. Help? > It should be possible to indicate that the transmitted data comes > from several places in the bid. This allows the OPES processor to > omit huge cookies and other junk; the response, by including this > information, helps the process limit the state and parsing. Interesting. If I interpret your requirement correctly, we need an indication that some data was skipped by the OPES processor when forwarding original application message to OPES server. We also need an ability to reinject the skipped data into produced application message. Not sure this can be supported in a general way: OPES server may modify application headers but it is not clear how it can tell OPES processor to correctly inject skipped stuff into modified headers if the server does not know exactly what was skipped. We may be able to support the above for, say, header values but not for header names (but this becomes too application-specific!). Alternatively, we can document that OPES processor is responsible for injecting skipped stuff the way it deems necessary. In the latter case, OPES server should be informed that something was skipped, but it would not care much except for message length interpretation code. This still makes digital signatures and related concepts hard to implement or verify. Am I making it more complicated than it is? Comments? Thank you, Alex. -- | HTTP performance - Web Polygraph benchmark www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite | all of the above - PolyBox appliance From owner-ietf-openproxy@mail.imc.org Wed Feb 19 10:50:56 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA29479 for ; Wed, 19 Feb 2003 10:50:52 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JFL7N21994 for ietf-openproxy-bks; Wed, 19 Feb 2003 07:21:07 -0800 (PST) Received: from zrc2s0jx.nortelnetworks.com (zrc2s0jx.nortelnetworks.com [47.103.122.112]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JFL5d21989 for ; Wed, 19 Feb 2003 07:21:05 -0800 (PST) Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1JFKrg00566; Wed, 19 Feb 2003 09:20:53 -0600 (CST) Received: by zsc3c028.us.nortel.com with Internet Mail Service (5.5.2653.19) id <1LWH4W82>; Wed, 19 Feb 2003 07:20:53 -0800 Message-ID: <0A11633F61BD9F40B43ABCC694004F93F18DA5@zsc3c026.us.nortel.com> From: "Reinaldo Penno" To: "'Alex Rousskov'" , ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft Date: Wed, 19 Feb 2003 07:20:48 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D82A.7B2A7EA6" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D82A.7B2A7EA6 Content-Type: text/plain Before I make any comments, it seems to me this work has a lot of overlap with things we already did. 1. The examples are greatly covered in the use cases and deployment scenarios 2. Some other things seems to me that belong in the requirements draft. It doesn't make any sense to have two drafts having requirements on the protocol, moreover the sole purpose of one of them was to have all the requirements. 3. The idea was to try to reuse a existing protocol instead of designing a new one. 4. There are two protocols for OPES, in-path and the callout. Are they going to be the same, different? I guess what we need is to incorporate whatever requirements in a bis/whatever version of the requirements draft, extend the use cases and scenarios and for every protocol we think has a chance to be the OPES protocol, match its semantics against the requirements draft, that's what a requirement draft is for. If Alex's document is the bootstrap for the following deliverable "MAY 03 Initial protocol document for OPES services including their authorization, invocation, tracking, and enforcement of authorization." I guess it should be much more to the point and focused. Should we hold a conf call to iron these things out? Chairs? I guess we should be also working on the rules specification? Regards, Reinaldo -----Original Message----- From: Alex Rousskov [mailto:rousskov@measurement-factory.com] Sent: Wednesday, February 19, 2003 12:57 AM To: ietf-openproxy@imc.org Subject: Re: OPES protocol, pre-draft On Tue, 18 Feb 2003, The Purple Streak, Hilarie Orman wrote: > The OPES processor should make the decisions about what to send to the > consumer. This might just be a matter of terminology, but the > processor is in control of the source and destination and should not > send messages to new destinations based on OPES server demands. It looks like a design decision to me (i.e., it can be done both ways). I think that you may want destination modification by OPES servers if you want to satisfy Martin's requirement to be able to produce multiple SMTP messages (to several small groups of recipients) from one original SMTP message (to one large group of recipients). My understanding is that it MAY be OPES server responsibility to "split" the original destination address(es) into two groups. That is why I let OPES server to modify original destination info. Other destination-modification examples include request redirection (within a CDN or at the surrogate). Source-modification examples include anonymizing proxies. If there is a consensus that OPES server cannot modify source and destination info, then we can simplify the protocol a little bit. Is there a consensus regarding this design decision? Perhaps Abbie's poll will show... > The start message should identify the total length of the data, if it > is available. This might stretch over several "bids", see below. Good point. This extra info about anticipated message length should not hurt and may be used for resource pre-allocation purposes. It MUST NOT be treated as normative/final, of course. I will modify the messages accordingly. > The first response from the server should identify the new total > length, if it is available. If the length will change, but the new > size is unknown, the server should indicate this. Agreed, except we also need to support the case where the OPES server does not know whether the length will change. For example, if the server replaces "foo" with "blah" and "bleh" with "bar", it can tell the final length (and whether it will change) only after seeing all message content. To make things more general and symmetric, I would make it possible (but optional) to supply this non-normative length estimate with every relevant message, in both directions. > There is some confusion about "destination" - the OPES server should > never change the destination (i.e., the endpoint), so I don't see why > it is needed. In the redirection example, it would be sufficient to > change the headers, and the purpose of "destination" is a mystery to > me. See above for motivation. The destination is needed because the OPES processor needs to know where to connect to forward the request. It is Bad Design to have OPES processor guess that information from [possibly modified] message headers. This, again, assumes that we want OPES server to be able to modify destination addresses. If we do not want that, there is no need for OPES server to pass that info back to processor, of course. Note that source and destination information is meta-level information that is often not completely available from HTTP headers. Take interception and WCCP-controlled proxies for example. These intermediaries often have to get destination address based on IP-level details, not from HTTP headers. Similarly, the source information is usually not available in the request headers but may be required to route and modify the message. Moreover, the protocol should make it possible to exchange other meta-level information. For example, the time of the request may be important ("no porn surfing before 6pm!"). > The relationship between the application-layer framing, the bid and > offset, and the OPES framing is not clear from your examples. The > application data may be transmitted to the OPES server a packet at a > time - this will mean a different bid on every data message, if you > literally mean that a bid is a buffer id. Otherwise, it should have > some other name. Yes, a terminology/naming problem: By "buffer" in Buffer ID you probably mean "piece of memory that holds a data packet". I meant "logical structure that holds all data associated with the application connection/message". That is, my-buffer may consist of many your-buffers. Perhaps "buffer" should be replaced with "connection"? But "connection" is bad because OPES server does not really manage application connections. "Message" seems too overloaded? "Application message" (amid)? Will change bid to amid unless there are better ideas. This is just terminology though. "Bid/amid" is permanent for the single application message (original or produced). This ID should be used by processor and server to manage appropriate data structures associated with the corresponding application connection. > Also, the information about a bid should include its total length. Not sure why that would be needed. Moreover, the "total length" of the connection buffers (which is what bid identifies) may change at any time. OPES server should not care how buffering is done at the OPES processor side and vice versa. Perhaps your suggestion is a result of my poor choice of the word "buffer"? See above. > It should be possible to send the start and end messages on a separate > transport connection for handling errors or congestion. Yes, and it is possible. The start message is the first message for an "amid" so it can go on any connection (brand new or idle persistent). The end messages, if they indicate an immediate abort, do not have to be in order with data messages and, hence, can be sent on any OPES connection as well. Recall that there is no protocol-mandated relation between OPES connections and application transactions/connections. If one wants to sent something "out of order", they can (and face the consequences). > The case discussed for multiple services, multiple responses, should > be included. To support it, one needs multiple service lists and an > id for each response. I believe this is already supported, kind of. The "services" attribute of the transaction start message can have a list of services. The OPES server can initiate multiple consumer-start messages based on that list. Each consumer-start message from OPES server has a unique bid (amid). What is not clear to me is how the OPES server would know whether the services list is an OR, AND, or XOR, or something else. I suspect we need to support If-header logic from ICAP if we want to go down this route. I will polish the protocol once I understand the exact requirement here. Do we need to support some kind of service-to-response matching language here? I do not recall a clear answer in available OPES IDs. Help? > It should be possible to indicate that the transmitted data comes from > several places in the bid. This allows the OPES processor to omit > huge cookies and other junk; the response, by including this > information, helps the process limit the state and parsing. Interesting. If I interpret your requirement correctly, we need an indication that some data was skipped by the OPES processor when forwarding original application message to OPES server. We also need an ability to reinject the skipped data into produced application message. Not sure this can be supported in a general way: OPES server may modify application headers but it is not clear how it can tell OPES processor to correctly inject skipped stuff into modified headers if the server does not know exactly what was skipped. We may be able to support the above for, say, header values but not for header names (but this becomes too application-specific!). Alternatively, we can document that OPES processor is responsible for injecting skipped stuff the way it deems necessary. In the latter case, OPES server should be informed that something was skipped, but it would not care much except for message length interpretation code. This still makes digital signatures and related concepts hard to implement or verify. Am I making it more complicated than it is? Comments? Thank you, Alex. -- | HTTP performance - Web Polygraph benchmark www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite | all of the above - PolyBox appliance ------_=_NextPart_001_01C2D82A.7B2A7EA6 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES protocol, pre-draft

Before I make any comments, it seems to me this work = has a lot of overlap with things we already did.

1. The examples are greatly covered in the use cases = and deployment scenarios
2. Some other things seems to me that belong in the = requirements draft. It doesn't make any sense to have two drafts having = requirements on the protocol, moreover the sole purpose of one of them = was to have all the requirements.

3. The idea was to try to reuse a existing protocol = instead of designing a new one.
4. There are two protocols for OPES, in-path and the = callout. Are they going to be the same, different?

I guess what we need is to incorporate whatever = requirements in a bis/whatever version of the requirements draft, = extend the use cases and scenarios and for every protocol we think has = a chance to be the OPES protocol, match its semantics against the = requirements draft, that's what a requirement draft is for.

If Alex's document is the bootstrap for the following = deliverable

"MAY 03 Initial protocol document for OPES = services including their authorization, invocation, tracking, and = enforcement of authorization."

I guess it should be much more to the point and = focused. Should we hold a conf call to iron these things out?

Chairs?

I guess we should be also working on the rules = specification?


Regards,

Reinaldo

-----Original Message-----
From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
Sent: Wednesday, February 19, 2003 12:57 AM
To: ietf-openproxy@imc.org
Subject: Re: OPES protocol, pre-draft



On Tue, 18 Feb 2003, The Purple Streak, Hilarie Orman = wrote:

> The OPES processor should make the decisions = about what to send to the
> consumer.  This might just be a matter of = terminology, but the
> processor is in control of the source and = destination and should not
> send messages to new destinations based on OPES = server demands.

It looks like a design decision to me (i.e., it can = be done both ways). I think that you may want destination modification = by OPES servers if you want to satisfy Martin's requirement to be able = to produce multiple SMTP messages (to several small groups of = recipients) from one original SMTP message (to one large group of = recipients). My understanding is that it MAY be OPES server = responsibility to "split" the original destination = address(es) into two groups. That is why I let OPES server to modify = original destination info.

Other destination-modification examples include = request redirection (within a CDN or at the surrogate). = Source-modification examples include anonymizing proxies.

If there is a consensus that OPES server cannot = modify source and destination info, then we can simplify the protocol a = little bit. Is there a consensus regarding this design decision? = Perhaps Abbie's poll will show...

> The start message should identify the total = length of the data, if it
> is available.  This might stretch over = several "bids", see below.

Good point. This extra info about anticipated message = length should not hurt and may be used for resource pre-allocation = purposes. It MUST NOT be treated as normative/final, of course. I will = modify the messages accordingly.

> The first response from the server should = identify the new total
> length, if it is available.  If the length = will change, but the new
> size is unknown, the server should indicate = this.

Agreed, except we also need to support the case where = the OPES server does not know whether the length will change. For = example, if the server replaces "foo" with "blah" = and "bleh" with "bar", it can tell the final length = (and whether it will change) only after seeing all message = content.

To make things more general and symmetric, I would = make it possible (but optional) to supply this non-normative length = estimate with every relevant message, in both directions.

> There is some confusion about = "destination" - the OPES server should
> never change the destination (i.e., the = endpoint), so I don't see why
> it is needed.  In the redirection example, = it would be sufficient to
> change the headers, and the purpose of = "destination" is a mystery to
> me.

See above for motivation. The destination is needed = because the OPES processor needs to know where to connect to forward = the request. It is Bad Design to have OPES processor guess that = information from [possibly modified] message headers. This, again, = assumes that we want OPES server to be able to modify destination = addresses. If we do not want that, there is no need for OPES server to = pass that info back to processor, of course.

Note that source and destination information is = meta-level information that is often not completely available from HTTP = headers. Take interception and WCCP-controlled proxies for example. = These intermediaries often have to get destination address based on = IP-level details, not from HTTP headers. Similarly, the source = information is usually not available in the request headers but may be = required to route and modify the message.

Moreover, the protocol should make it possible to = exchange other meta-level information. For example, the time of the = request may be important ("no porn surfing before = 6pm!").

> The relationship between the application-layer = framing, the bid and
> offset, and the OPES framing is not clear from = your examples.  The
> application data may be transmitted to the OPES = server a packet at a
> time - this will mean a different bid on every = data message, if you
> literally mean that a bid is a buffer id.  = Otherwise, it should have
> some other name.

Yes, a terminology/naming problem: By = "buffer" in Buffer ID you probably mean "piece of memory = that holds a data packet". I meant "logical structure that = holds all data associated with the application = connection/message".  That is, my-buffer may consist of many = your-buffers. Perhaps "buffer"  should be replaced with = "connection"? But "connection" is bad because OPES = server does not really manage application connections. = "Message" seems too overloaded? "Application = message" (amid)? Will change bid to amid unless there are better = ideas.

This is just terminology though. "Bid/amid" = is permanent for the single application message (original or produced). = This ID should be used by processor and server to manage appropriate = data structures associated with the corresponding application = connection.

> Also, the information about a bid should include = its total length.

Not sure why that would be needed. Moreover, the = "total length" of the connection buffers (which is what bid = identifies) may change at any time. OPES server should not care how = buffering is done at the OPES processor side and vice versa. Perhaps = your suggestion is a result of my poor choice of the word = "buffer"? See above.

> It should be possible to send the start and end = messages on a separate
> transport connection for handling errors or = congestion.

Yes, and it is possible. The start message is the = first message for an "amid"  so it can go on any = connection (brand new or idle persistent). The end messages, if they = indicate an immediate abort, do not have to be in order with data = messages and, hence, can be sent on any OPES connection as well. Recall = that there is no protocol-mandated relation between OPES connections = and application transactions/connections. If one wants to sent = something "out of order", they can (and face the = consequences).

> The case discussed for multiple services, = multiple responses, should
> be included.  To support it, one needs = multiple service lists and an
> id for each response.

I believe this is already supported, kind of. The = "services" attribute of the transaction start message can = have a list of services. The OPES server can initiate multiple = consumer-start messages based on that list. Each consumer-start message = from OPES server has a unique bid (amid).

What is not clear to me is how the OPES server would = know whether the services list is an OR, AND, or XOR, or something = else. I suspect we need to support If-header logic from ICAP if we want = to go down this route. I will polish the protocol once I understand the = exact requirement here. Do we need to support some kind of = service-to-response matching language here? I do not recall a clear = answer in available OPES IDs. Help?

> It should be possible to indicate that the = transmitted data comes from
> several places in the bid.  This allows = the OPES processor to omit
> huge cookies and other junk; the response, by = including this
> information, helps the process limit the state = and parsing.

Interesting. If I interpret your requirement = correctly, we need an indication that some data was skipped by the OPES = processor when forwarding original application message to OPES server. = We also need an ability to reinject the skipped data into produced = application message. Not sure this can be supported in a general way: = OPES server may modify application headers but it is not clear how it = can tell OPES processor to correctly inject skipped stuff into modified = headers if the server does not know exactly what was = skipped.

We may be able to support the above for, say, header = values but not for header names (but this becomes too = application-specific!). Alternatively, we can document that OPES = processor is responsible for injecting skipped stuff the way it deems = necessary. In the latter case, OPES server should be informed that = something was skipped, but it would not care much except for message = length interpretation code. This still makes digital signatures and = related concepts hard to implement or verify. Am I making it more = complicated than it is? Comments?


Thank you,

Alex.

--
          &nb= sp;           &nb= sp;     | HTTP performance - Web Polygraph = benchmark www.measurement-factory.com | HTTP compliance+ - Co-Advisor = test suite

          &nb= sp;           &nb= sp;     | all of the above - PolyBox = appliance

------_=_NextPart_001_01C2D82A.7B2A7EA6-- From owner-ietf-openproxy@mail.imc.org Wed Feb 19 11:11:56 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA00038 for ; Wed, 19 Feb 2003 11:11:53 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JFgwr23210 for ietf-openproxy-bks; Wed, 19 Feb 2003 07:42:58 -0800 (PST) Received: from sccrmhc01.attbi.com (sccrmhc01.attbi.com [204.127.202.61]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JFgod23205 for ; Wed, 19 Feb 2003 07:42:50 -0800 (PST) Received: from oskar3 (h00a0c5e5337d.ne.client2.attbi.com[24.218.184.133]) by sccrmhc01.attbi.com (sccrmhc01) with SMTP id <200302191542460010069rlce>; Wed, 19 Feb 2003 15:42:46 +0000 From: "Oskar Batuner" To: "Alex Rousskov" , Subject: RE: OPES protocol, pre-draft Date: Wed, 19 Feb 2003 10:43:19 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit 1. Terminology: I'd suggest using names "OPES processor" and "callout server", as in the architecture document. OPES server is a bit confusing - too similar to OPES processor (which in fact is also a server). I'd also suggest using transaction-id (tid) instead of buffer-id. 2. Transaction boundaries: in your example OPES server does not send xaction-end. This may result in protocol stack on the callout server keeping some tid-related data for indefinite time - in order to correctly match possible incoming messages to the protocol state. I'd suggest keeping clear transaction structure: each side MUST open and close transaction with predefined start-end messages. This will result in more stable and reliable protocol state machine. 3. Client-server protocol versus independent message flow. I think that client-server architecture (as in ICAP and HTTP) has certain advantages. It fits well OPES dataflow model: OPES server always initiates transaction as result of some message received from customer. Callout server provides services that are initiated by request made by OPES processor. This structure is simpler, and if we need transactions initiated by callout server independent from OPES data flow (for example request for policy information, address book, preferences, etc.) we may use a different unidirectional channel. In our case by-directional channel may be presented as two unidirectional ones, and as far as I can see OPES protocol fits client-server model very well. All exchanges are taking place in the context of some transaction, each such transaction consist of request-response sequences, in each transaction we have a clear division of roles: one side is a service requestor (client), another - a service produce (server), and the protocol messages that each side may produce are subordinate to it's role in the current transaction. Moreover, there are distinct sets of requests that may be send by OPES server / callout server. - Oskar > -----Original Message----- > From: owner-ietf-openproxy@mail.imc.org > [mailto:owner-ietf-openproxy@mail.imc.org]On Behalf Of Alex Rousskov > Sent: Tuesday, February 18, 2003 6:25 PM > To: ietf-openproxy@imc.org > Subject: OPES protocol, pre-draft > > > > > To bootstrap creation of OPES protocol specs and to show one possible > way to answer the 5 design questions raised on this list, I would like > to propose the following protocol outline. If you think this is too > premature to talk about protocol specifics, please suggest what must > be done first. > > I believe this protocol core can be implemented on top of HTTP or > BEEP. I am less sure about SOAP; I am not a SOAP expert. > > I think that the relaxed flow of the proposed protocol is a good idea, > given the combination of robustness requirements and requirements to > support a diverse set of application protocols. Mandatory ACKs and > explicit client-server flow do little good when you are managing > several semi-independent I/O buffers and have to handle, for example, > OPES server disappearance/overload and multiple recipients of a single > message. This relaxed flow and focus on data manipulation are the key > design elements that make the protocol simple yet efficient and robust > (I hope). > > The protocol is optimized for the common case when most data is > forwarded "as is", unaffected by the OPES server. Removing this > optimization can simplify the protocol further. > > > -------------------- > > Contents: > > 1. Terminology and environment > 2. OPES processor-server communication > 2.1 Transaction-independent messages > 2.2 Transaction-dependent messages > 2.2.1 Common transaction-dependent properties > 2.2.2 Major message types > 3. Examples > 4. Transport connections > 5. Synchronization and error handling > > 1. Terminology and environment > > draft-ietf-opes-protocol-reqs-03.txt defines the following information > flow: > > data provider --(original application message)--> > -- [ OPES magic ] --> > --(produced application messages)--> data consumers > > The original and "produced" (forwarded) messages together form an > application protocol transaction. Note that there may be more > than one produced application message resulting from a single > original message. > > When application protocol transaction involves a request-response > sequence (e.g., HTTP), the above scheme remains the same. There are > just two related transactions now: > > provider1 --(client req)--> [ OPES ] --(proxy req)--> consumer1 > provider2 --(server resp)-> [ OPES ] --(proxy resp)-> consumer2 > > Usually, provider1 is consumer2, and consumer1 is provider2. > > Multiple related transactions do not change the nature of the > OPES protocol. The only difference is that more information may > need to be passed from OPES processor to OPES server. For > example, processing of the response flow may need some knowledge > about the request flow. Individual transactions may become > related. > > There may be no server response. Depending on the application > protocol, there may be multiple server and/or proxy responses. > > > > 2. OPES processor-server communication > > OPES processor and OPES server exchange messages. The exchange is > bidirectional. There is no clear client or server role. There is > no clear request/response message category either. > > There are two major kinds of messages: transaction-independent > and transaction-dependent. > > > 2.1 Transaction-independent messages > > Transaction-independent messages exchange information unrelated > to any specific application protocol transaction. They are used > to enquire about supported OPES protocol options, control remote > logging, exchange user preferences, check connectivity status, > etc. Transaction-independent messages are not documented here. > > > 2.2 Transaction-dependent messages > > Transaction-dependent messages are the core of the OPES protocol. > These messages alter, block, redirect, clone application messages > to be forwarded to data consumers. Each transaction-dependent > message is associated with a single application transaction by > means of a unique transaction identifier (xid). > > It may be important to keep in mind that transaction-dependent > messages manipulate the state of these four buffers/connections and > associated meta-information: > - data producer (incoming) buffer at the OPES processor > - data producer (incoming) buffer at the OPES server > - data consumer (outgoing) buffer at the OPES server > - data consumer (outgoing) buffer at the OPES processor > > The design keeps buffers as allows to prevent buffer overflows > and discard buffered content when possible. Note that we rely on > transport protocol to be both reliable and to stop sending us > more data (eventually) if we stop reading it. TCP has both > properties. > > > 2.2.1 Common transaction-dependent properties > > Some transaction-dependent OPES messages share the following common > properties. > > xid -- Application transaction identifier (Xaction ID) > > source -- Information about the data provider (i.e., the > source of the application message). For messages > originated from the OPES processor, the source describes > the original data provider. For messages originated from > the OPES server, the source describes what provider > information should be presented to the data consumer; > OPES server may need to change how the original > information looks to the other application side. > > destinations -- One or more destinations. A single > destination is information about the data consumer (i.e, > the destination of the application message). For messages > originated from the OPES processor, destination describes > the consumer as intended by the producer. For messages > originated from the OPES server, the destination is the > data consumer that should be used by the OPES processor; > OPES server may need to change the intended recipient. > Depending on the application, OPES processor may need > to check that all destinations have been covered by > OPES server. > > data size -- Specific data size in octets OR a special token > meaning "all" or "maximum". The all-token may only be > used when requesting data, never when sending it. > > reason -- This should probably be a numeric status code with > an optional information string. We will use just strings > for now. > > > 2.2.2 Major message types > > OPES processor may send the following transaction-dependent > messages to the OPES server. [ Note: The XML-like syntax does NOT > imply that the messages should be implemented using XML. Text or > binary encodings can be used; the encoding decision is out of > scope of this document. ] > > > Informs OPES server about a new application > transaction. This message should probably identify OPES > service(s) requested for this transaction and other > transaction-global info unrelated to data buffering, > sources, or destinations. > > > Informs OPES server about a new request from the data > producer. Buffer IDentifier (bid) uniquely identifies the data > buffer used for this request. Bid can probably be set to > xid unless we expect to handle protocols that may merge > requests before forwarding them. > > > Sends [a portion of] application message from the data > producer buffer to the OPES server. If "copied" flag > is set, the OPES server may assume that the corresponding > data is buffered at the processor and may refer to it > using messages described below. > > > Notifies OPES server that there will be no more data > for this transaction (coming from the OPES processor) > UNLESS OPES server explicitly asks for it using > message described below > > > Notifies OPES server that there will be no more data > for this transaction (coming from the OPES processor) > > > Notifies OPES server that there will be no more messages > for this bid (coming from the OPES processor) > > > Notifies OPES server that there will be no more messages > for this transaction (coming from the OPES processor) > > > OPES server may send the following transaction-dependent > messages to the OPES processor. > > > Informs OPES processor that OPES server may want to send > data from source to destination(s). The Buffer IDentifier > (bid) is unique for the (xid, source, destinations) > triplet and identifies consumer buffer at the OPES > server. There may be other buffers/messages (bid > triplets) associated with the same transaction (xid). Xid > comes from the corresponding xaction-start message send > by the OPES processor. > > > Tells OPES processor to send the attached data to the > data consumer > > > Tells OPES processor to use processor's own copy of the > specified data to send to the data consumer. This message > can only specify data fragments previously marked with > "copied" flag in a message from OPES processor. > > > Tells OPES processor that the server will never send > data-asis message for the specified data range. This > message can only specify data fragments previously marked > with the "copied" flag in a message from OPES > processor. This optional message may help OPES processor > to free its resources. > > > Tells OPES processor to send the specified data segment > to the OPES server (probably in response to data-pause > message from the OPES server). > > > Notifies OPES processor that it should not send more data > for this transaction until OPES server explicitly asks > for it using data-need message described above > > > Tells OPES processor that there will be no more data > for this bid (coming from the OPES server) > > > Notifies OPES server that there will be no more messages > for this bid (coming from the OPES server) > > > Notifies OPES server that there will be no more messages > for this transaction (coming from the OPES server) > > > Note: There needs to be a way for OPES server to tell OPES > processor to terminate (or short-circuit) the forwarding of a > message. This feature needs to be added to the protocol, but it > should not change the overall design. > > > 3. Examples > > Here is an example of (not) filtering an HTTP message based > on HTTP headers: > > processor: > processor: > processor: > processor: > > server: > server: > server: > > Note that xaction-end implies consumer-end implies data-end, and > there is no reason for OPES processor to send a xaction-end > message to server if the server already sent xaction-end message. > The lines above are grouped about possible network I/O > boundaries; thus, only two network data packets may be required > to process a message if the OPES server decides it does not care > based on the headers. > > > Here is an example of redirecting an HTTP request by changing its > destination info and corresponding HTTP headers: > > processor: > processor: > processor: > processor: > > server: > server: > server: > > > Finally, here is an example of modifying the "middle" part of > HTTP message body. The OPES server switches the message encoding > to chunked, to avoid buffering data to figure out new Content-Length. > > processor: > processor: > processor: > processor: > > server: > server: > server: > server: > > processor: > > server: > > processor: > > /* send modified chunk, tell processor to ignore the original */ > server: > server: > > processor: > processor: > > server: > server: > > Note that once the flow starts, there are no explicit synchronization > points or waiting. The above message order is not the only one > possible: most messages from the processor are not synchronized with > most messages from the server. > > > 4. Transport connections > > Transport connections would depend on the transport protocol > (HTTP, BEEP, etc.). It is important to note that regardless of > the transport protocol chosen, it is possible to multiplex > messages from the OPES processor (or from the server) over > several persistent connections -- transaction-dependent messages > do not depend on "connection" properties except for the basic > requirement that ordered messages use the same connection, in the > right order. > > Simple connection-specific messages can be introduced if we want > to support keep-alive checks or if we want to retry aborted > connections. > > > 5. Synchronization and error handling. > > The protocol has very few explicit dependencies between messages. > It is trivial to imagine a case where incorrect processor or > server implementation would result in deadlocks or other bad > states. All sorts of deadlocks are resolved using timeouts. If > there is no progress with the transaction for an > admin-configurable time, the transaction is aborted. Aborting at > OPES server side is easy: > > server: > > On the processor side, specific actions would depend on the > protocol and state. For example, if no response bytes have been > sent to an HTTP client yet, then an error response can be sent. > > It would be also possible, in some states, to eliminate OPES > server from processing if it fails. Supporting this behavior > would require having a copy of entire application messages even > is OPES server tells us it does not need a copy. The exact > behavior must be admin-configurable. > > > --------------------------- > > > This is just a start, of course. Many details are not specified > yet. For example, can OPES server request that a sub-fragment of > copied fragment is forwarded "as is"? What is the best way to > handle modifications of a non-chunked HTTP transfer that change > total message-length? > > Also note that many command/option names should probably be > changed/polished. It is quite possible, for example, to make the > protocol look like the next generation of ICAP if we want to. > > > I believe the proposed protocol covers current major ICAP capabilities > and supports a few desired optimizations mentioned on the mailing > list. Is there anything major that the protocol core does not support > and that needs to be supported? (besides security and authentication > features that are irrelevant at this point). > > For example, should we add commands to control persistency of > application-protocol connections from the OPES server (I do not think > so)? Should we make it possible for OPES server to change the > application protocol (I do not think so)? > > > Thank you, > > Alex. > > -- > | HTTP performance - Web Polygraph benchmark > www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite > | all of the above - PolyBox appliance From owner-ietf-openproxy@mail.imc.org Wed Feb 19 12:15:35 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA01895 for ; Wed, 19 Feb 2003 12:15:30 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JGmae25346 for ietf-openproxy-bks; Wed, 19 Feb 2003 08:48:36 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JGmYd25342 for ; Wed, 19 Feb 2003 08:48:34 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1JGmaeM031918 for ; Wed, 19 Feb 2003 09:48:36 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1JGmaJe031917; Wed, 19 Feb 2003 09:48:36 -0700 (MST) (envelope-from rousskov) Date: Wed, 19 Feb 2003 09:48:36 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Wed, 19 Feb 2003, Oskar Batuner wrote: > 1. Terminology: I'd suggest using names "OPES processor" and > "callout server", as in the architecture document. OPES server is a > bit confusing - too similar to OPES processor (which in fact is also > a server). Will change. > I'd also suggest using transaction-id (tid) instead of buffer-id. Transaction ID is already used to identify application transaction (xid). Xid is [the only thing] shared among messages from and to OPES processor. "Buffer ID" clearly causes confusion though. Do you like Application Message ID (amid) instead of buffer-id? An application transaction usually involves several application messages (original and produced), which is reflected in how "amid" (currently "bid") is used by the protocol. > 2. Transaction boundaries: in your example OPES server does not send > xaction-end. This may result in protocol stack on the callout server > keeping some tid-related data for indefinite time - in order to > correctly match possible incoming messages to the protocol state. > > I'd suggest keeping clear transaction structure: each side MUST open > and close transaction with predefined start-end messages. This will > result in more stable and reliable protocol state machine. The text should probably be polished to make it more clear. There is absolutely NO reason for the side that receives a xaction-end message to send a xaction-end message back. Here is why: - a side MUST ignore messages for unknown transactions (except for xaction-start messages, of course); doing so requires keeping state for known transactions, which is something that has to be done anyway; a simple hash table would do - once a side sends a xaction-end message, it deletes all corresponding data structures, and the corresponding xaction ID immediately becomes "unknown" for that side - if the other side responds to the xaction-end message with its own xaction-end message, the latter will simply be ignored by the first side; sending a xaction-end message to the side that already sent xaction-end message is a waste of resources As you can see, the state machine is deterministic and simple. Once a side says that it does not care about the transaction, there is no reason to tell it _anything_ about that transaction. There is no way a correct implementation can end up with xid-related data for indefinite time because of the above optimizations -- there is no xid-related data at the side once xaction-end message is sent by that side. Does this clarify? Did I miss anything? > 3. Client-server protocol versus independent message flow. I think > that client-server architecture (as in ICAP and HTTP) has certain > advantages. It fits well OPES dataflow model: OPES server always > initiates transaction as result of some message received from > customer. Callout server provides services that are initiated by > request made by OPES processor. This structure is simpler, First of all, there is a terminology problem here (possibly introduced by my own wording!). The proposed protocol uses a "pipeline" model. It is still possible to call it a client-server protocol, I guess. The key difference is the difference between strict "at most one-outstanding-query" approach and a "pipeline of queries". But let's use the client-server term to describe "at most one-outstanding-query" approach, for now. I do not see how a client-server model is simpler. Can you elaborate? The proposed "pipeline" protocol has a very straightforward (simple?) state-machine-like implementation. Moreover, I argue that client-server model is inappropriate for OPES because it does not handle data flow efficiently. It does not scale with the size of the application message. Client-server model is good for handling small atomic transactions. The proposed pipeline model is good for handling data transfers. If we want OPES to be efficient then we should optimize data flow, and client-server model does not allow for that. Client-server abstraction is OK when working on high-level and treating each application message as a size-less blob of information: Time Action ---- ------ 0 Read the original application message 1 Send query: "Please modify this message" 2 Receive response: "Here is a modified message" 3 Write the produced application message Once we start looking at the implementation level, the above becomes very inefficient both because of the resources spend keeping all the state and because of the introduced delays. This traditional approach does not scale with the message size. ICAP tries to solve the problem with the "preview" feature, but that solution is partial and rigid. To get rid of large buffers and staging delays one must use a pipeline approach: Time Action ---- ------ t Read chunk #N of the original message t Modify chunk #N-1 of the original message t Write chunk #N-2 of the original message t+1 Send query: "Please modify chunk #N" t+1 Receive response: "Here is modified chunk #N-1 ... Since most of the actions are performed on small chunks and in parallel, the buffering requirements are minimal and so is the extra delay. Response time overhead of a good OPES implementation should not exceed that of a [second] proxy overhead. Historical evidence in point: Optimized HTTP proxies do not use client-server model internally. The proxies pipeline data from the client to server side and back. OPES is also a proxy (just look at the architecture diagrams in the release WG IDs). > and if we need transactions initiated by callout server independent > from OPES data flow (for example request for policy information, > address book, preferences, etc.) we may use a different > unidirectional channel. Note that the proposed protocol does not define transaction-related messages that are spontaneously initiated by callout server, independent from OPES data flow. The callout server always "responds" to the stream of messages initiated by the OPES processor. As for transaction-independent messages, yes, those can be implemented on separate channels. Having separate channels may make priority handling of transaction-dependent messages easier. We can discuss this (less important design decision) later. > In our case by-directional channel may be presented as two > unidirectional ones, and as far as I can see OPES protocol fits > client-server model very well. All exchanges are taking place in the > context of some transaction, each such transaction consist of > request-response sequences, in each transaction we have a clear > division of roles: one side is a service requestor (client), another > - a service produce (server), and the protocol messages that each > side may produce are subordinate to it's role in the current > transaction. Moreover, there are distinct sets of requests that may > be send by OPES server / callout server. I mostly agree. As I said, there seems to be a terminology problem. The proposed approach fits your description above relatively well; the key [difference] is that the request-response pairs are not sequential but pipelined. Without the pipeline, the protocol does not scale with the message size (see illustrations above). Once you accept the pipeline idea, everything should snap into place. In other words, the proposed protocol is closed to a traditional client-server approach at the level of message _chunks_. The important part is, however, not handling of individual chunks, but making an efficient pipeline from chunk-related "queries" and "responses". Do you agree? Can you think of a pipeline-less alternative that still scales well with the message size? Thank you, Alex. -- | HTTP performance - Web Polygraph benchmark www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite | all of the above - PolyBox appliance From owner-ietf-openproxy@mail.imc.org Wed Feb 19 12:23:35 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA02092 for ; Wed, 19 Feb 2003 12:23:29 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JGuRE25971 for ietf-openproxy-bks; Wed, 19 Feb 2003 08:56:27 -0800 (PST) Received: from hosting.altserver.com ([209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JGuQd25964 for ; Wed, 19 Feb 2003 08:56:26 -0800 (PST) Received: from f05v-13-4.d1.club-internet.fr ([212.194.192.4] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18lXWM-0008Oi-00 for ietf-openproxy@imc.org; Wed, 19 Feb 2003 08:56:35 -0800 Message-Id: <5.2.0.9.0.20030219180150.02e37820@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Wed, 19 Feb 2003 18:01:57 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: Re: OPES protocol, pre-draft Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-48A6288; boundary="=======52D9264=======" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=======52D9264======= Content-Type: text/plain; x-avg-checked=avg-ok-48A6288; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit On 06:56 19/02/03, Alex Rousskov said: >If there is a consensus that OPES server cannot modify source and >destination info, then we can simplify the protocol a little bit. >Is there a consensus regarding this design decision? Perhaps Abbie's >poll will show... No. The OPES can sub-delegate a task temporarily or permanently. The first task that a server must be able to do is to qualify a request as a dispatcher. We all know an application which is based on that thinking: the DNS. jfc --=======52D9264======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-48A6288 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.454 / Virus Database: 253 - Release Date: 10/02/03 --=======52D9264=======-- From owner-ietf-openproxy@mail.imc.org Wed Feb 19 12:23:58 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA02111 for ; Wed, 19 Feb 2003 12:23:56 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JGuQT25962 for ietf-openproxy-bks; Wed, 19 Feb 2003 08:56:26 -0800 (PST) Received: from hosting.altserver.com ([209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JGuPd25955 for ; Wed, 19 Feb 2003 08:56:25 -0800 (PST) Received: from f05v-13-4.d1.club-internet.fr ([212.194.192.4] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18lXWJ-0008Oi-00 for ietf-openproxy@imc.org; Wed, 19 Feb 2003 08:56:31 -0800 Message-Id: <5.2.0.9.0.20030219175929.02e37e00@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Wed, 19 Feb 2003 18:01:34 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: RE: SMTP filtering use case Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-48A6288; boundary="=======4D2D777D=======" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=======4D2D777D======= Content-Type: multipart/alternative; x-avg-checked=avg-ok-48A6288; boundary="=====================_5955507==.ALT" --=====================_5955507==.ALT Content-Type: text/plain; x-avg-checked=avg-ok-48A6288; charset=iso-8859-1; format=flowed Content-Transfer-Encoding: 8bit On 01:01 19/02/03, Abbie Barbir said: >I would like to make a summary of yes and no and maybe's for the protocol >that could be (after further enhancements) presented in SFO. Here is a quick dispatcher side list of questions I try to address right now with "C" functions and real machines in mind. I read this list to try to get ideas/responses, so I have not yet formalized a list of questions to address, so it is only a few and in disorder. Different thninking it seems. If it may help? - is it (dispatcher) meant to be single? or to work in: - coordination (one is master) - cooperation (related egal) - distribution (lose equal / specialized ) etc. - can it delegate the dispatching role? - on load - by default (failure) - on rules - how - what is the inter-dispatcher protocol - does it include axfr or each dispatcher is by its own? etc. - which servers does it relate with? - how the servers letit know they are there - how does it négotiate cooperation - how is that information memorized? - how is it maintained (refreshed)? - what are the recovery procedures - are there connection/task priorities supported? - is competing support supported? - are there some task recovery support? - type of channel/security support - how is the user to remotely contol the ONES server - information it is to pass? - when? - TTL? - dissemination? When? - reporting to the user? - cancellation/zapping ability - negociations etc. - how do we want to write the specifications of the protocol? - plain language - specialiized language - specific high level language adapted to the architecture - or work on the definitions as we progress - "C" examples as function or a reference default OPES dispatcher and servers. jfc --=====================_5955507==.ALT Content-Type: text/html; x-avg-checked=avg-ok-48A6288; charset=iso-8859-1 Content-Transfer-Encoding: 8bit On 01:01 19/02/03, Abbie Barbir said:
I would like to make a summary of yes and no and maybe's for the protocol that could be (after further enhancements) presented in SFO.

Here is a quick dispatcher side list of questions I try to address right now with "C" functions and real machines in mind.  I read this list to try to get ideas/responses, so I have not yet formalized a list of questions to address, so it is only a few and in disorder. Different thninking it seems. If it may help?

  - is it (dispatcher) meant to be single? or to work in:
     - coordination (one is master)
     - cooperation (related egal)
     - distribution (lose equal / specialized )
     etc.

  - can it delegate the dispatching role?
     - on load
     - by default (failure)
     - on rules
     - how
     - what is the inter-dispatcher protocol
     - does it include axfr or each dispatcher is by its own?
     etc.

  - which servers does it relate with?
    - how the servers letit know they are there
    - how does it négotiate cooperation
    - how is that information memorized?
    - how is it maintained (refreshed)?
    - what are the recovery procedures
    - are there connection/task priorities supported?
    - is competing support supported?
    - are there some task recovery support?
    - type of channel/security support
 
- how is the user to remotely contol the ONES server
  - information it is to pass?
  - when?
  - TTL?
  - dissemination? When?
  - reporting to the user?
  - cancellation/zapping ability
  - negociations
    etc.
        
- how do we want to write the specifications of the protocol?
   - plain language
   - specialiized language
   - specific high level language adapted to the architecture
   - or work on the definitions as we progress
   - "C" examples as function or a reference default
      OPES dispatcher and servers.
 
jfc

 --=====================_5955507==.ALT-- --=======4D2D777D======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-48A6288 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.454 / Virus Database: 253 - Release Date: 10/02/03 --=======4D2D777D=======-- From owner-ietf-openproxy@mail.imc.org Wed Feb 19 12:58:19 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA03209 for ; Wed, 19 Feb 2003 12:58:15 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JHXjV28059 for ietf-openproxy-bks; Wed, 19 Feb 2003 09:33:45 -0800 (PST) Received: from hosting.altserver.com ([209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JHXhd28055 for ; Wed, 19 Feb 2003 09:33:44 -0800 (PST) Received: from f05v-13-4.d1.club-internet.fr ([212.194.192.4] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18lY6P-0005xl-00 for ietf-openproxy@imc.org; Wed, 19 Feb 2003 09:33:50 -0800 Message-Id: <5.2.0.9.0.20030219182411.02e6d7b0@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Wed, 19 Feb 2003 18:38:50 +0100 To: From: jfcm Subject: RE: OPES protocol, pre-draft In-Reply-To: References: Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-48A6288; boundary="=======274A34DA=======" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=======274A34DA======= Content-Type: text/plain; x-avg-checked=avg-ok-48A6288; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit At 16:43 19/02/03, Oskar Batuner wrote: >1. Terminology: I'd suggest using names "OPES processor" and "callout server", >as in the architecture document. OPES server is a bit confusing - too similar >to OPES processor (which in fact is also a server). > >I'd also suggest using transaction-id (tid) instead of buffer-id. Might we not think in term of dispatcher and services? Does location of these functions change the logic of the relations between the dispatcher (applying the filtering rules) and the service(s)? >2. Transaction boundaries: in your example OPES server does not send >xaction-end. This may result in protocol stack on the callout server keeping >some tid-related data for indefinite time - in order to correctly match >possible incoming messages to the protocol state. > >I'd suggest keeping clear transaction structure: each side MUST open and close >transaction with predefined start-end messages. This will result in more >stable and reliable protocol state machine. I am not sure about this: can matters subject to rules of the filter be changed by the server? Canit result in loops? >3. Client-server protocol versus independent message flow. I think that >client-server architecture (as in ICAP and HTTP) has certain advantages. It >fits well OPES dataflow model: OPES server always initiates transaction as >result of some message received from customer. not if it acts as a sub-server. Question can we hide that sub-servicing? Exemple: the OPES server must add a documentation inclduing a graphic. That graphic mustbe obtained from another paying OPES server. There is a need for the sub-transaction to be recorded (payement, legal responsbility etc). >Callout server provides >services that are initiated by request made by OPES processor. This structure >is simpler, and if we need transactions initiated by callout server >independent from OPES data flow (for example request for policy information, >address book, preferences, etc.) we may use a different unidirectional >channel. bi-directionnal if we want to have negociations? jfc --=======274A34DA======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-48A6288 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.454 / Virus Database: 253 - Release Date: 10/02/03 --=======274A34DA=======-- From owner-ietf-openproxy@mail.imc.org Wed Feb 19 13:15:31 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA03814 for ; Wed, 19 Feb 2003 13:15:25 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JHpIF29676 for ietf-openproxy-bks; Wed, 19 Feb 2003 09:51:18 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JHpGd29670 for ; Wed, 19 Feb 2003 09:51:16 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1JHpIeM033491 for ; Wed, 19 Feb 2003 10:51:18 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1JHpI0B033490; Wed, 19 Feb 2003 10:51:18 -0700 (MST) (envelope-from rousskov) Date: Wed, 19 Feb 2003 10:51:18 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft In-Reply-To: <0A11633F61BD9F40B43ABCC694004F93F18DA5@zsc3c026.us.nortel.com> Message-ID: References: <0A11633F61BD9F40B43ABCC694004F93F18DA5@zsc3c026.us.nortel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Wed, 19 Feb 2003, Reinaldo Penno wrote: > Before I make any comments, it seems to me this work has a lot of > overlap with things we already did. > > 1. The examples are greatly covered in the use cases and deployment > scenarios The examples (Section 3) simply illustrate how to map the protocol commands to the use cases. > 2. Some other things seems to me that belong in the requirements > draft. It doesn't make any sense to have two drafts having > requirements on the protocol, moreover the sole purpose of one of > them was to have all the requirements. It felt awkward to simply send a dry protocol spec draft without any motivation/introduction/examples. I guess it is safe to ignore those sections if you can grok the protocol spec without them. On the other hand, the final OPES protocol RFC will have to duplicate a lot of information from current drafts. Implementors and integrators will use that RFC as a primary reference point. They should not be required to merge three or four RFCs/IDs to understand and implement the protocol. In other words, the requirements drafts are for IETF (our) internal use; the protocol RFC is for the world. I am also not sure my proposal contains any new requirements. It contains beginnings of a protocol to satisfy existing requirements. > 3. The idea was to try to reuse a existing protocol instead of > designing a new one. We SHOULD reuse. So far, I have not seen any existing protocol that would meet the requirements and support desirable features mentioned on this list. Did I miss any? Moreover, as far as I can see, there is only one protocol, ICAP, that is "an existing OPES-like protocol". Other things like HTTP/BEEP/SOAP mentioned on this list are lower-level abstractions that can only be used as a foundation of an OPES protocol (like ICAP uses HTTP). As I noted, the proposed protocol can be implemented on top of HTTP, BEEP, and (maybe) SOAP. One of the reasons I made the proposal is so that the discussion "ICAP or something new" becomes more substantiated. I expect ICAP proponents defend ICAP on this list. ICAP (or any other public deployed protocol) should be given a "bonus point" compared to anything new, of course. > 4. There are two protocols for OPES, in-path and the callout. Are > they going to be the same, different? What document describes the in-path protocol? I based my proposal on draft-ietf-opes-protocol-reqs-03.txt which talks about one [callout] protocol only and does not mention the "in-path" protocol. Also, draft-ietf-opes-architecture-04.txt specifically says in Section 7 that OPES ruleset schema and OPES callout protocol are necessary and sufficient OPES elements. It does not mention the "in-path" protocol. What am I missing? > I guess what we need is to incorporate whatever requirements in a > bis/whatever version of the requirements draft, extend the use cases > and scenarios I agree with the latter (use cases and scenarios). It looks like there were a few desirable features mentioned on this list but not documented in the drafts (e.g., creating multiple responses based on a single message). I did not hear any new requirements, but I probably missed them. > and for every protocol we think has a chance to be the > OPES protocol, match its semantics against the requirements draft, > that's what a requirement draft is for. My understanding is that the above has been done for ICAP (draft-stecher-opes-icap-eval-00.txt). Again, what are other protocols to be considered? > If Alex's document is the bootstrap for the following deliverable > > "MAY 03 Initial protocol document for OPES services including their > authorization, invocation, tracking, and enforcement of authorization." Yes, I guess it is. > I guess it should be much more to the point and focused. See my lame excuses above. The only section that might survive (after lots of editing, of course) is section 2 "OPES processor-server communication", which is already as focused as it can be. > Should we hold a conf call to iron these things out? I would be happy to participate though the mailing list seems to be working pretty well so far. Thank you, Alex. -- | HTTP performance - Web Polygraph benchmark www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite | all of the above - PolyBox appliance From owner-ietf-openproxy@mail.imc.org Wed Feb 19 13:18:29 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA03872 for ; Wed, 19 Feb 2003 13:18:25 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JHlja29128 for ietf-openproxy-bks; Wed, 19 Feb 2003 09:47:45 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JHlid29122 for ; Wed, 19 Feb 2003 09:47:44 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1JHlkeM033386 for ; Wed, 19 Feb 2003 10:47:46 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1JHlkU1033385; Wed, 19 Feb 2003 10:47:46 -0700 (MST) (envelope-from rousskov) Date: Wed, 19 Feb 2003 10:46:54 -0700 (MST) From: Alex Rousskov To: jfcm Subject: Re: OPES protocol, pre-draft In-Reply-To: <5.2.0.9.0.20030219180732.02e35cb0@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030219130759.02e12bd0@mail.utel.net> <200302190230.h1J2UFn13195@localhost.localdomain> <200302190230.h1J2UFn13195@localhost.localdomain> <5.2.0.9.0.20030219130759.02e12bd0@mail.utel.net> <5.2.0.9.0.20030219180732.02e35cb0@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: jfc, We are talking about two different things here. You are talking about directing application messages to appropriate callout servers. I am talking about [re]directing application messages to different data consumers. For example, an HTTP request for www.w3c.com can be redirected (by the callout server manipulations of HTTP headers and meta-information) to www.w3.org. Another example: an e-mail to managers@example.com can be redirected (by the callout server manipulations of SMTP headers and meta-information) to ceo@example.com _and_ to officers@example.com (producing two application messages). Some argued that such modification of application message destination MUST NOT be allowed. Some wanted it to support the second example above. Hope this clarifies. Thank you, Alex. On Wed, 19 Feb 2003, jfcm wrote: > On 17:53 19/02/03, Alex Rousskov said: > > > No. > > - "No. There is no consensus: I think that a callout server > MAY change the destination of the application transaction, > but some other folks on the list disagree." > > I do not like the word transation in here. I prefer cession. It can be a > lot more structured that just a transaction. Unless my Frenglish hits again. > > > > The OPES > > Server or Dispatcher system > > > > can sub-delegate a task temporarily or permanently. The > > > first task that a server must be able to do is to qualify a request > > > as a dispatcher. > > The Dispatcher qualifies a page as to be processed > > 1. it can send it to another/sub-dispatcher which is in charge > of handling that type of service (ex. DNS) > > 2. it can send it to a callout server which may decide > that this type of service will be handled by another > machine on the Internet or CPU in its own network. > > And this several times if loops are permitted on rules > (the process permits to change something subject to > a rule) or even if several rules are to be applied in > sequence (but is that the same transaction according > to your vision?). > > Thank you. > jfc > > From owner-ietf-openproxy@mail.imc.org Wed Feb 19 13:31:24 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA04172 for ; Wed, 19 Feb 2003 13:31:22 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JIAns02709 for ietf-openproxy-bks; Wed, 19 Feb 2003 10:10:49 -0800 (PST) Received: from hosting.altserver.com ([209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JIAmd02705 for ; Wed, 19 Feb 2003 10:10:48 -0800 (PST) Received: from f05v-13-4.d1.club-internet.fr ([212.194.192.4] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18lYgK-00028x-00 for ietf-openproxy@imc.org; Wed, 19 Feb 2003 10:10:56 -0800 Message-Id: <5.2.0.9.0.20030219185235.02f38200@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Wed, 19 Feb 2003 19:16:43 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: RE: OPES protocol, pre-draft In-Reply-To: References: Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-48A6288; boundary="=======532559E2=======" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=======532559E2======= Content-Type: text/plain; x-avg-checked=avg-ok-48A6288; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit At 17:48 19/02/03, Alex Rousskov wrote: >Once you accept the pipeline idea, everything should snap into place. total agreement. Takes good shape! Now, in term of wording why pipes would they not link "dispatchers" and end into "services" (or service admins - which could include a dispatcher)? An OPES domain would then end at services admins (which could belong to several domains). This architecture can be supported off the shelf in any message passing (network) protocol. 1. user data flows go through the first dispatcher 2. filters it to outboud socket or to a pipe 3. receiving dispatcher triggers the proper service admin 4. service admin tells the dispatcher which service to pipe it into 5. anytime the service can call the disptacher to get additional services 6. the output is sent back to first dispatcher From what you say there would be "data" pipes and "command" pipes, the command pipes being faster? Reason is that there is a need for load management and therefore priorities. Pipes should permit that nicely but the sending dispatcher needs to control the throttle. If you use a pipeline approach, would you support zappers in the pipes? Reason of the importance of that is that if a service blocks, the preceding dispatcher must be able to zap the outband pipe and rebuild it in calling a new service admin. jfc --=======532559E2======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-48A6288 Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.454 / Virus Database: 253 - Release Date: 10/02/03 --=======532559E2=======-- From owner-ietf-openproxy@mail.imc.org Wed Feb 19 13:42:03 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA04446 for ; Wed, 19 Feb 2003 13:42:01 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JIJqr03037 for ietf-openproxy-bks; Wed, 19 Feb 2003 10:19:52 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JIJpd03033 for ; Wed, 19 Feb 2003 10:19:51 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1JIJdt25773; Wed, 19 Feb 2003 13:19:40 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6V4V2>; Wed, 19 Feb 2003 13:19:40 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404EC9362@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: "Reinaldo Penno" , "'Alex Rousskov'" , ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft Date: Wed, 19 Feb 2003 13:19:37 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D843.76204530" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D843.76204530 Content-Type: text/plain reinaldo, see inline, abbie -----Original Message----- From: Penno, Reinaldo [BL60:0430:EXCH] Sent: Wednesday, February 19, 2003 10:21 AM To: 'Alex Rousskov'; ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft Before I make any comments, it seems to me this work has a lot of overlap with things we already did. 1. The examples are greatly covered in the use cases and deployment scenarios 2. Some other things seems to me that belong in the requirements draft. It doesn't make any sense to have two drafts having requirements on the protocol, moreover the sole purpose of one of them was to have all the requirements. --- abbie Yes, this is whay at some point i did state that 3. The idea was to try to reuse a existing protocol instead of designing a new one. well, this discussion will then lead to ok, what can we use if this is the protocol that we have just designed. So, it does not hurt to do what we are doing, actually it may be healthy. The best way to figure out if we can reuse something is to design a protocol first and then say, woo it really looks like ICAP or BEEP or ..... (SOAP) 4. There are two protocols for OPES, in-path and the callout. Are they going to be the same, different? focus now on callout protocol. I guess what we need is to incorporate whatever requirements in a bis/whatever version of the requirements draft, extend the use cases and scenarios and for every protocol we think has a chance to be the OPES protocol, match its semantics against the requirements draft, that's what a requirement draft is for. If Alex's document is the bootstrap for the following deliverable "MAY 03 Initial protocol document for OPES services including their authorization, invocation, tracking, and enforcement of authorization." I guess it should be much more to the point and focused. Should we hold a conf call to iron these things out? Chairs? I guess we should be also working on the rules specification? ------abbie why not?????? Regards, Reinaldo SNIP ------_=_NextPart_001_01C2D843.76204530 Content-Type: text/html Message
reinaldo,
see inline,
 
abbie
 
-----Original Message-----
From: Penno, Reinaldo [BL60:0430:EXCH]
Sent: Wednesday, February 19, 2003 10:21 AM
To: 'Alex Rousskov'; ietf-openproxy@imc.org
Subject: RE: OPES protocol, pre-draft

Before I make any comments, it seems to me this work has a lot of overlap with things we already did.

1. The examples are greatly covered in the use cases and deployment scenarios
2. Some other things seems to me that belong in the requirements draft. It doesn't make any sense to have two drafts having requirements on the protocol, moreover the sole purpose of one of them was to have all the requirements. 

 

--- abbie

Yes, this is whay at some point i did state that 

3. The idea was to try to reuse a existing protocol instead of designing a new one.  

well, this discussion will then lead to ok, what can we use if this is the protocol that we have just designed. So, it does not hurt to do what we are doing, actually it may be healthy. The best way to figure out if we can reuse something is to design a protocol first and then say, woo it really looks like ICAP or BEEP or ..... (SOAP)

 
4. There are two protocols for OPES, in-path and the callout. Are they going to be the same, different?  

focus now on callout protocol.  

I guess what we need is to incorporate whatever requirements in a bis/whatever version of the requirements draft, extend the use cases and scenarios and for every protocol we think has a chance to be the OPES protocol, match its semantics against the requirements draft, that's what a requirement draft is for.

If Alex's document is the bootstrap for the following deliverable

"MAY 03 Initial protocol document for OPES services including their authorization, invocation, tracking, and enforcement of authorization."

I guess it should be much more to the point and focused. Should we hold a conf call to iron these things out?

Chairs?

I guess we should be also working on the rules specification?  

------abbie

why not?????? 


Regards,

Reinaldo

 SNIP

 

------_=_NextPart_001_01C2D843.76204530-- From owner-ietf-openproxy@mail.imc.org Wed Feb 19 13:47:11 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA04583 for ; Wed, 19 Feb 2003 13:47:07 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JINTA03136 for ietf-openproxy-bks; Wed, 19 Feb 2003 10:23:29 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JINTd03132 for ; Wed, 19 Feb 2003 10:23:29 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1JINIt26121; Wed, 19 Feb 2003 13:23:18 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6V4YH>; Wed, 19 Feb 2003 13:23:19 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404EC9377@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft Date: Wed, 19 Feb 2003 13:23:15 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D843.F8474446" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D843.F8474446 Content-Type: text/plain hi, good discussion so far. see inline abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Wednesday, February 19, 2003 12:57 AM > To: ietf-openproxy@imc.org > Subject: Re: OPES protocol, pre-draft > SNIP > If there is a consensus that OPES server cannot modify source > and destination info, then we can simplify the protocol a > little bit. Is there a consensus regarding this design > decision? Perhaps Abbie's poll will show... > --- abbie well, it seems we do not have one, so here, please i would like direct votes. Chairs, Can u please remark on this requirement??????? For the recored, I think that OPES should be able to abbie SNIP ------_=_NextPart_001_01C2D843.F8474446 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES protocol, pre-draft

hi,

good discussion so far.

see inline

abbie

> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Wednesday, February 19, 2003 12:57 = AM
> To: ietf-openproxy@imc.org
> Subject: Re: OPES protocol, pre-draft
>
SNIP

> If there is a consensus that OPES server cannot = modify source
> and destination info, then we can simplify the = protocol a
> little bit. Is there a consensus regarding this = design
> decision? Perhaps Abbie's poll will = show...
>

--- abbie
well, it seems we do not have one, so here, please i = would like direct votes.

Chairs, Can u please remark on this = requirement???????

For the recored, I think that OPES should be able = to


abbie



SNIP
 

------_=_NextPart_001_01C2D843.F8474446-- From owner-ietf-openproxy@mail.imc.org Wed Feb 19 14:23:01 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA05780 for ; Wed, 19 Feb 2003 14:23:00 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JJ1aU03994 for ietf-openproxy-bks; Wed, 19 Feb 2003 11:01:36 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JJ1Yd03989 for ; Wed, 19 Feb 2003 11:01:34 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1JJ1ZeM036110 for ; Wed, 19 Feb 2003 12:01:35 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1JJ1ZXR036109; Wed, 19 Feb 2003 12:01:35 -0700 (MST) (envelope-from rousskov) Date: Wed, 19 Feb 2003 12:01:35 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft In-Reply-To: <87609AFB433BD5118D5E0002A52CD75404EC9377@zcard0k6.ca.nortel.com> Message-ID: References: <87609AFB433BD5118D5E0002A52CD75404EC9377@zcard0k6.ca.nortel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Wed, 19 Feb 2003, Abbie Barbir wrote: > > If there is a consensus that OPES server cannot modify source > > and destination info, then we can simplify the protocol a > > little bit. Is there a consensus regarding this design > > decision? Perhaps Abbie's poll will show... > > well, it seems we do not have one, so here, please i would like > direct votes. > > Chairs, Can u please remark on this requirement??????? > > For the recored, I think that OPES should be able to For the record, I also think that OPES should be able to. I think so because (a) it allows for broader manipulations and (b) some of those manipulations would be possible (in an awkward way) anyway, tempting violators. The security/privacy part of the protocol should be especially strong when it comes to source/destination changes, of course. An example for the item (b) above. Consider two HTTP proxies that are chained together. The first proxy on the request path has an OPES callout server attached. A request for origin server A arrives at the first proxy. The callout server modifies HTTP headers so that the request is now destined to a different origin server B. The modified request is forwarded to the second proxy (the first proxy is not aware of the destination changes). The second proxy is not OPES-aware. It simply forwards the request to B. Thus, if I use two proxies, I can implicitly modify request destination using OPES! If this can be hacked, we would be better off allowing (and controlling) it explicitly. Alex. From owner-ietf-openproxy@mail.imc.org Wed Feb 19 14:38:20 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA06320 for ; Wed, 19 Feb 2003 14:38:17 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JJFon04313 for ietf-openproxy-bks; Wed, 19 Feb 2003 11:15:50 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JJFnd04309 for ; Wed, 19 Feb 2003 11:15:49 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1JJFct07356; Wed, 19 Feb 2003 14:15:38 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6VWD7>; Wed, 19 Feb 2003 14:15:38 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404EC9439@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft Date: Wed, 19 Feb 2003 14:15:38 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D84B.49833E80" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D84B.49833E80 Content-Type: text/plain agreed, abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Wednesday, February 19, 2003 2:02 PM > To: ietf-openproxy@imc.org > Subject: RE: OPES protocol, pre-draft > > > > On Wed, 19 Feb 2003, Abbie Barbir wrote: > > > > If there is a consensus that OPES server cannot modify source and > > > destination info, then we can simplify the protocol a > little bit. Is > > > there a consensus regarding this design decision? Perhaps Abbie's > > > poll will show... > > > > well, it seems we do not have one, so here, please i would > like direct > > votes. > > > > Chairs, Can u please remark on this requirement??????? > > > > For the recored, I think that OPES should be able to > > For the record, I also think that OPES should be able to. I > think so because (a) it allows for broader manipulations and > (b) some of those manipulations would be possible (in an > awkward way) anyway, tempting violators. > > The security/privacy part of the protocol should be > especially strong when it comes to source/destination > changes, of course. > > An example for the item (b) above. Consider two HTTP proxies > that are chained together. The first proxy on the request > path has an OPES callout server attached. A request for > origin server A arrives at the first proxy. The callout > server modifies HTTP headers so that the request is now > destined to a different origin server B. The modified request > is forwarded to the second proxy (the first proxy is not > aware of the destination changes). The second proxy is not > OPES-aware. It simply forwards the request to B. Thus, if I > use two proxies, I can implicitly modify request destination > using OPES! If this can be hacked, we would be better off > allowing (and controlling) it explicitly. > > Alex. > > ------_=_NextPart_001_01C2D84B.49833E80 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES protocol, pre-draft

agreed,

abbie


> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Wednesday, February 19, 2003 2:02 = PM
> To: ietf-openproxy@imc.org
> Subject: RE: OPES protocol, pre-draft
>
>
>
> On Wed, 19 Feb 2003, Abbie Barbir wrote:
>
> > > If there is a consensus that OPES = server cannot modify source and
> > > destination info, then we can = simplify the protocol a
> little bit. Is
> > > there a consensus regarding this = design decision? Perhaps Abbie's
> > > poll will show...
> >
> > well, it seems we do not have one, so = here, please i would
> like direct
> > votes.
> >
> > Chairs, Can u please remark on this = requirement???????
> >
> > For the recored, I think that OPES should = be able to
>
> For the record, I also think that OPES should = be able to. I
> think so because (a) it allows for broader = manipulations and
> (b) some of those manipulations would be = possible (in an
> awkward way) anyway, tempting violators.
>
> The security/privacy part of the protocol = should be
> especially strong when it comes to = source/destination
> changes, of course.
>
> An example for the item (b) above. Consider two = HTTP proxies
> that are chained together. The first proxy on = the request
> path has an OPES callout server attached. A = request for
> origin server A arrives at the first proxy. The = callout
> server modifies HTTP headers so that the = request is now
> destined to a different origin server B. The = modified request
> is forwarded to the second proxy (the first = proxy is not
> aware of the destination changes). The second = proxy is not
> OPES-aware. It simply forwards the request to = B. Thus, if I
> use two proxies, I can implicitly modify = request destination
> using OPES! If this can be hacked, we would be = better off
> allowing (and controlling) it = explicitly.
>
> Alex.
>
>

------_=_NextPart_001_01C2D84B.49833E80-- From owner-ietf-openproxy@mail.imc.org Wed Feb 19 15:24:47 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA07728 for ; Wed, 19 Feb 2003 15:24:45 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JJt8K05766 for ietf-openproxy-bks; Wed, 19 Feb 2003 11:55:08 -0800 (PST) Received: from hosting.altserver.com ([209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JJt7d05762 for ; Wed, 19 Feb 2003 11:55:07 -0800 (PST) Received: from f04a-6-104.d1.club-internet.fr ([212.194.77.104] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18laIm-0003Wu-00 for ietf-openproxy@imc.org; Wed, 19 Feb 2003 11:54:45 -0800 Message-Id: <5.2.0.9.0.20030219205729.02476c00@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Wed, 19 Feb 2003 20:57:36 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: Re: OPES protocol, pre-draft Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-752115EC; boundary="=======3E407969=======" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=======3E407969======= Content-Type: text/plain; x-avg-checked=avg-ok-752115EC; charset=us-ascii; format=flowed Content-Transfer-Encoding: 8bit On 18:46 19/02/03, Alex Rousskov said: >jfc, > >We are talking about two different things here. You are >talking about directing application messages to appropriate callout >servers. I am talking about [re]directing application messages to >different data consumers. > >For example, an HTTP request for www.w3c.com can be redirected (by the >callout server manipulations of HTTP headers and meta-information) to >www.w3.org. Another example: an e-mail to managers@example.com can be >redirected (by the callout server manipulations of SMTP headers and >meta-information) to ceo@example.com _and_ to officers@example.com >(producing two application messages). Oh! sorry. Just followed my own idea and I did not make it clear enough about transaction/cession. I do not make any difference between OPES processor and server and possibly clients. See my other mail. I consider dispatchers and services. I responded about the server changing or additing direction as simpler. I am only cautious about telling I do want support your both cases, because I do not know which loop control you may have been discussed already. Also, I am glad you want to support smtp I was rebuked not to consider http only. Let assume that www.w3c.com says to redirect calls to www.w3.org and that www.w3.org says to redirect calls to www.w3c.com. What happens? But I certainly want that that type of service (I am not really familliar with Internet protocols): is there a trick to achieve that today? - I would love it for an application - it would be a good case to say that OPES would make it a service rather than a trick. My understanding of the client as part of the OPES system is because the client may interact with the OPES server. That traffic must be left open by the OPES processor and it muts have top priority. Example: in this redirection case, the callout server sends back a question to the client: "this call will be rerouted are you OK?". The OPES may also be a gateway asking for an ID/password. Or a make a translation in an other language and to need some more information to perform (ex. the meaning of a word in a menu). Thank you. jfc --=======3E407969======= Content-Type: text/plain; charset=us-ascii; x-avg=cert; x-avg-checked=avg-ok-752115EC Content-Disposition: inline --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.454 / Virus Database: 253 - Release Date: 10/02/03 --=======3E407969=======-- From owner-ietf-openproxy@mail.imc.org Wed Feb 19 15:42:15 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA08308 for ; Wed, 19 Feb 2003 15:42:15 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JKJ1f07419 for ietf-openproxy-bks; Wed, 19 Feb 2003 12:19:01 -0800 (PST) Received: from linux.grifflink.com (linux.royaleng.com [216.83.131.67]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JKJ0d07415 for ; Wed, 19 Feb 2003 12:19:00 -0800 (PST) Received: from localhost.localdomain ([192.168.6.22]) by linux.grifflink.com (8.12.5/8.12.5) with ESMTP id h1JKpbnT025978; Wed, 19 Feb 2003 13:51:38 -0700 Received: (from ho@localhost) by localhost.localdomain (8.11.6/8.11.6) id h1JKJgx29382; Wed, 19 Feb 2003 13:19:42 -0700 Date: Wed, 19 Feb 2003 13:19:42 -0700 Message-Id: <200302192019.h1JKJgx29382@localhost.localdomain> From: "The Purple Streak, Hilarie Orman" To: abbieb@nortelnetworks.com Cc: ietf-openproxy@imc.org In-reply-to: Yourmessage <87609AFB433BD5118D5E0002A52CD75404EC9439@zcard0k6.ca.nortel.com> Subject: RE: OPES protocol, pre-draft Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: If you are going to allow the callout processor to have that much control, then it should be a fully general engine and be able to select the transport protocol, port numbers, application protocol, etc. Heck, it should be able to open the connections itself. I'd presented this concept at one of the pre-WG meetings, as an additional work item that might go onto a charter someday, but there was no interest. It's still a fine idea, but not for this WG. The very focused approach of in-the-flow content adaptation that we have now is a good one, and it reflects current practice and engineering principles that are known to work very efficiently. In this model, the communication control point is the OPES box, and it is not slaved to the callout servers. The OPES rules can be used to change the source and destination, and the OPES box is the most efficient place to accomplish that function. Spreading the essential control functions over a set of callout servers sets up all kinds of configuration nightmares that will hinder adoption of OPES. Hilarie From owner-ietf-openproxy@mail.imc.org Wed Feb 19 16:05:07 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA08841 for ; Wed, 19 Feb 2003 16:05:06 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JKbF009066 for ietf-openproxy-bks; Wed, 19 Feb 2003 12:37:15 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JKbEd09058 for ; Wed, 19 Feb 2003 12:37:14 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1JKbGeM038294 for ; Wed, 19 Feb 2003 13:37:16 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1JKbGWi038293; Wed, 19 Feb 2003 13:37:16 -0700 (MST) (envelope-from rousskov) Date: Wed, 19 Feb 2003 13:37:16 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: Re: OPES protocol, pre-draft Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Wed, 19 Feb 2003, jfcm wrote: > Let assume that www.w3c.com says to redirect calls to www.w3.org and > that www.w3.org says to redirect calls to www.w3c.com. What happens? A loop detection code at the second hop would see its own Via: header already in the HTTP headers of the request and would not forward the request further, responding with an error. This is how redirections (and loop detection) are supposed to work today. This issue is not directly related to OPES except that OPES callout server might participate in loop detection by adding its own Via headers (or application-specific equivalent). I do not know whether the current requirement/architecture drafts consider callout server a "proxy hop" on the application protocol path. They probably should. > But I certainly want that that type of service (I am not really > familliar with Internet protocols): is there a trick to achieve that > today? You can certainly do the redirections today (any decent HTTP intermediary can do that). Moreover, you can already do pretty much anything on the traffic-modification level that OPES promises to deliver! The reason we work on OPES is to provide a common interface for "traffic modifiers" _and_ to address IAB concerns about them. > My understanding of the client as part of the OPES system is because > the client may interact with the OPES server. That traffic must be > left open by the OPES processor and it muts have top priority. > > Example: in this redirection case, the callout server sends back a > question to the client: "this call will be rerouted are you OK?". > The OPES may also be a gateway asking for an ID/password. Or a make > a translation in an other language and to need some more information > to perform (ex. the meaning of a word in a menu). My understanding is that OPES processor and callout server may negotiate with each other, but may not negotiate with the data producer or data consumer. Or, in other words, such "external" negotiations are outside of OPES scope. However, OPES may document application-protocol-specific(?) ways to [statically] specify data producer's or data consumer's OPES preferences when making application protocol transactions. For example, a web browser may attach a "never translate the response" OPES preference to an HTTP request, and that preference would get to the OPES processor and/or callout server if they are in the path. Alex. -- | HTTP performance - Web Polygraph benchmark www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite | all of the above - PolyBox appliance From owner-ietf-openproxy@mail.imc.org Wed Feb 19 17:18:35 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA11002 for ; Wed, 19 Feb 2003 17:18:34 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1JLmeB11251 for ietf-openproxy-bks; Wed, 19 Feb 2003 13:48:40 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1JLmcd11247 for ; Wed, 19 Feb 2003 13:48:38 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1JLmeeM039924 for ; Wed, 19 Feb 2003 14:48:40 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1JLmeTE039923; Wed, 19 Feb 2003 14:48:40 -0700 (MST) (envelope-from rousskov) Date: Wed, 19 Feb 2003 14:48:40 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft In-Reply-To: <200302192019.h1JKJgx29382@localhost.localdomain> Message-ID: References: <200302192019.h1JKJgx29382@localhost.localdomain> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Wed, 19 Feb 2003, The Purple Streak, Hilarie Orman wrote: > If you are going to allow the callout processor to have that much > control, then it should be a fully general engine and be able to > select the transport protocol, port numbers, application protocol, > etc. Yes, the scope of the allowed callout server modifications is not obvious: - Limit to application message content/payload only? but modifying content often requires modifying headers! but content is sometimes passed in the headers! but e-mail applications require modifying destination addresses! - Limit to application message (headers+payload) only? but examples demonstrate that it is often possible to modify the destination address and even the protocol by modifying the message headers and tricking OPES processor into modifying connection-level properties; if tricks are possible we should make them explicit! - Limit to any changes within the same application protocol? but I want to convert FTP to HTTP! - Limit nothing? but can we address privacy/security concerns if there are no limits to what OPES can do? > Heck, it should be able to open the connections itself. I do not think we can technically prevent/prohibit that. Ideally, our goal should be NOT to prohibit all "bad" actions we can think of, but design the protocol that makes privacy/security violations detectable and traceable. We cannot control real-world implementations; they will violate our MUSTs if our MUSTs are too restrictive (HTTP is a good example). What we might be able to do is to enable data consumers and data producers to optionally detect and attribute violations of their preferences. For example: the client does not care whether a callout server opened a connection to doubleckick.com while processing an HTTP transaction. The client only cares that doubleckick.com does not insert an X-rated ad into the response. > I'd presented this concept at one of the pre-WG meetings, as an > additional work item that might go onto a charter someday, but there > was no interest. It's still a fine idea, but not for this WG. It seems within current WG scope though. Content adaptation may include "redirecting" content to a better "adapter". > The very focused approach of in-the-flow content adaptation that we > have now is a good one, and it reflects current practice and > engineering principles that are known to work very efficiently. True. That is why I originally asked whether there is consensus on this issue. If others are convinced by your arguments, I will adjust the protocol draft accordingly. > ... The OPES rules can be used to change the source and destination, > and the OPES box is the most efficient place to accomplish that > function. That's a very good point! However, I am not sure the current drafts allow what you want. draft-ietf-opes-architecture-04.txt says "OPES rules: these specify when and how to execute OPES services." This does not seem to include any modifications. draft-beck-opes-irml-02.txt does not seem to have any rules that can modify anything. The rules in that draft are only for invoking an appropriate OPES service. So, it looks like there is a second, related question to be asked. Do we want OPES rules to be able to modify something? For example, should OPES rules be able to modify destination of an application message? If we allow modification-by-rules but not by callout servers, how can an OPES system implement the following modifications? - if HTTP request contains an XSS (cross-site script), encode XSS to be HTML-safe, and change the destination URL to example.com/explain/XSS. - if incoming e-mail contains a virus, delete the virus and redirect the e-mail to staging@area.com; send a notification to the recipient As you can see, the above examples make destination modification dependent on content filtering/modification done by the callout server. Can/should OPES support the above? Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Fri Feb 21 06:51:02 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA26868 for ; Fri, 21 Feb 2003 06:51:00 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LBNfS24579 for ietf-openproxy-bks; Fri, 21 Feb 2003 03:23:41 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1LBNcd24570 for ; Fri, 21 Feb 2003 03:23:39 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id JD0IORVM; 21 Feb 2003 12:23:26 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: OPES protocol, pre-draft X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Fri, 21 Feb 2003 12:20:22 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C120@hermes.webwasher.com> Thread-Topic: OPES protocol, pre-draft Thread-Index: AcLYN9d518jrDgpoQv6y/NWq9oO/KQBXekvw From: "Martin Stecher" To: "Alex Rousskov" , Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1LBNdd24574 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit Hi Alex, hi all, > [...] > > Client-server abstraction is OK when working on high-level and > treating each application message as a size-less blob of information: > [...] > Once we start looking at the implementation level, the above becomes > very inefficient both because of the resources spend keeping all the > state and because of the introduced delays. This traditional approach > does not scale with the message size. ICAP tries to solve the problem > with the "preview" feature, but that solution is partial and rigid. To > get rid of large buffers and staging delays one must use a pipeline > approach: > [...] > Since most of the actions are performed on small chunks and in > parallel, the buffering requirements are minimal and so is the extra > delay. Response time overhead of a good OPES implementation should not > exceed that of a [second] proxy overhead. > ICAP/1.0 is already pipelined, I think. Messages are sent in chunked transfer encoding. ICAP clients send chunk N before waiting for the modified chunk N-1 to return. The important advantage of your protocol outline is that due to the introduction of xid and amid, it gets possible to implement the multiple preview feature even without waiting for a decision from the callout server before continueing. It is a full multiplex, asynchronous pipeline approach. No need to say: "I need more data" Easy to say: "I am done with this amid, as soon as you are able to go to the next application message, please do so" No "out-of-sync" problem I like this very much! Let me add two example here to show the difference between a classic approach and this one. Maybe it helps somebody as it helped me: Example 1: Callout server analyzes beginning of application message and detects that it wants to see and modify the complete application message: A. Traditional approach, ICAP/1.0 like with multiple preview 0. C: Start of app. message 1 1. C: Here is preview chunk 1 2. S: Need another preview chunk 3. C: Here is preview chunk 2 4. S: OK. I take all the rest 5. C: Chunk 3 of data 6. C: Chunk 4 of data S: mod. Chunks 1-3 of data 7. C: Chunk 5 of data S: mod. Chunk 4 of data 8. C: End of data S: mod. Chunk 5 of data 9. S: End of data 10. C: Start of app. message 2 B. Proposed OPES protocol 0. C: Start of app. message 1 1. C: Chunk 1 of data S: Start of app. message 1 2. C: Chunk 2 of data 3. C: Chunk 3 of data S: mod. Chunk 1-2 of data 4. C: Chunk 4 of data S: mod. Chunk 3 of data 5. C: Chunk 5 of data S: mod. Chunk 4 of data 6. C: End of data S: mod. Chunk 5 of data 7. C: Start of app. message 2 S: End of data So B has no waiting cycles before forwarding next chunks of data. But how does this change if the callout server decides after a second preview chunk that it is not interested in that data and wants the OPES processor to handle the (rest) of the file without forwarding it? Example 2: Callout server analyzes beginning of application message and detects that it is not interested in this app. message: A. Traditional approach, ICAP/1.0 like with multiple preview 0. C: Start of app. message 1 1. C: Here is preview chunk 1 2. S: Need another preview chunk 3. C: Here is preview chunk 2 4. S: Not interested (204 resp.) 5. C: Start of app. message 2 B. Proposed OPES protocol 0. C: Start of app. message 1 1. C: Chunk 1 of data S: Start of app. message 1 2. C: Chunk 2 of data 3. C: Chunk 3 of data S: Not interested 4. C: Chunk 4 of data 5. C: Start of app. message 2 In this case B sent more data and data chunks 3 and 4 will be ignored by the callout server. It receives data for an application message it is not (longer) working on. But due to amid which is send with all data packets too (not shown in the samples above) there is no danger to get out-of-sync. From owner-ietf-openproxy@mail.imc.org Fri Feb 21 07:21:00 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA27576 for ; Fri, 21 Feb 2003 07:20:59 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LBuK227364 for ietf-openproxy-bks; Fri, 21 Feb 2003 03:56:20 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1LBuJd27360 for ; Fri, 21 Feb 2003 03:56:19 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id N65U3D3D; 21 Feb 2003 12:56:16 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: OPES protocol, additional message needed? X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Fri, 21 Feb 2003 12:53:13 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C121@hermes.webwasher.com> Thread-Topic: OPES protocol, pre-draft Thread-Index: AcLXpgmCt0A/KqBdRxaj0uAd6TGkpwB9o9gg From: "Martin Stecher" To: Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1LBuJd27361 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit There is one common scenario with virus filtering in a callout server that I wrote about some time ago. Do you think an additional message type could help and is worth here? Scenario: A virus filter often cannot send chunks of data back to the OPES processor before it has the complete data. Only then it can start the scanning process. On the other hand it modifies data only in rare situations. Most results are "no virus found, use original message unmodified". Only if virus was found, it modifies (repair) the data or replaces it in parts or completly by an error message. So a possible message from callout server to OPES processor could be It informs the OEPS processor that the callout server is working on this although it does not reply data now. If "change-probability" is low, the OPES processor could use techniques to fight the raising latency time, for example: - some kind of download progress indication, - "data trickling" or - the proposed LateClearance content encoding (see draft-stecher-lclr-encoding-00.txt) The "progress" parameter can inform that the callout processor needs even more time although all data was already received (e.g. an archive needs to be extracted first which is a currently ongoing lengthy process) Thanks Martin From owner-ietf-openproxy@mail.imc.org Fri Feb 21 09:22:58 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA01135 for ; Fri, 21 Feb 2003 09:22:56 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LDwK500859 for ietf-openproxy-bks; Fri, 21 Feb 2003 05:58:20 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LDwId00854 for ; Fri, 21 Feb 2003 05:58:18 -0800 (PST) Received: from f12v-11-233.d1.club-internet.fr ([213.44.182.233] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18mDgv-0006Fw-00 for ietf-openproxy@imc.org; Fri, 21 Feb 2003 05:58:18 -0800 Message-Id: <5.2.0.9.0.20030221150123.02af8940@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Fri, 21 Feb 2003 15:03:44 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: RE: OPES protocol, pre-draft Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Sorry, I though I had sent back to Hillarie and the list - I hate these no reply to the list - quite confusing to be forced to talk privately on a public place. On 21:19 19/02/03, The Purple Streak, Hilarie Orman said: >If you are going to allow the callout processor to have that much >control, then it should be a fully general engine and be able to >select the transport protocol, port numbers, application protocol, >etc. Heck, it should be able to open the connections itself. I'd >presented this concept at one of the pre-WG meetings, as an additional >work item that might go onto a charter someday, but there was no >interest. It's still a fine idea, but not for this WG. Hillarie, my understanding of this WG is that IAB wants to have an http on-the-flow massaging stable solution, what they name OPES. Now I made clear that this is not only that I am interested in, but in what I name ONES (open network extended services). What I see from the mails is: 1. there is a clear understanding of what OPES are and how they could work in using SOAP, XML etc. CPU and line bandwidth consuming standard solutions. This is very interesting as it is a good consistent Internet response style with what is going around. It seems an exact, clear response to the IAB desire. 2. there is a smarter, pipe oriented, low cpu load approach with user interaction which corresponds to what I look for and which is actually like a network operating system. Maybe should we not confuse them and protect OPES from ONES in making two parallel propositions, one benefiting from the other? So developers could pick one or pick the other, instead of tricking OPES into ONES as they will probably want to do. And as we start ourselves doing. Because I think that if we do not respect the limitations imposed to OPES we will go all the way down to standalone engines as you describe. For example redirection could not be supported by OPES and total by ONES? >The very focused approach of in-the-flow content adaptation that we have >now is a good one, and it reflects current practice and engineering >principles that are known to work very efficiently. In this model, >the communication control point is the OPES box, and it is not slaved >to the callout servers. >The OPES rules can be used to change the source >and destination, and the OPES box is the most efficient place to accomplish >that function. This is a problem to me as I think dispatcher/service. This means that the OPENS box is a dispatcher (rules filtering) with a separated simple service? Or do you mean that the intermediary service is part of the rules? This has an impact on protocol because the same services could also be supported by a server should it become too heavy, etc.. >Spreading the essential control functions over a set of callout servers sets >up all kinds of configuration nightmares that will hinder adoption of >OPES. May be you are right. We want to have a transportation solution. IAB asked for a bicycle. Some think about a car. Better to think about a bicycle and about a car, rather than to try to build an Harley Davidson? jfc From owner-ietf-openproxy@mail.imc.org Fri Feb 21 10:56:31 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA04509 for ; Fri, 21 Feb 2003 10:56:29 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LFTOq07825 for ietf-openproxy-bks; Fri, 21 Feb 2003 07:29:24 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LFTNd07819 for ; Fri, 21 Feb 2003 07:29:24 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69] (may be forged)) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1LFTCa26488; Fri, 21 Feb 2003 10:29:12 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6WTDK>; Fri, 21 Feb 2003 10:29:12 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404F1DE65@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft Date: Fri, 21 Feb 2003 10:29:10 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D9BD.FADD87C6" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D9BD.FADD87C6 Content-Type: text/plain see inline, abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Wednesday, February 19, 2003 3:37 PM > To: ietf-openproxy@imc.org > Subject: Re: OPES protocol, pre-draft > > > > > On Wed, 19 Feb 2003, jfcm wrote: > > > Let assume that www.w3c.com says to redirect calls to > www.w3.org and > > that www.w3.org says to redirect calls to www.w3c.com. What happens? > > A loop detection code at the second hop would see its own > Via: header already in the HTTP headers of the request and > would not forward the request further, responding with an > error. This is how redirections (and loop detection) are > supposed to work today. > yes, in theory. > This issue is not directly related to OPES except that OPES > callout server might participate in loop detection by adding > its own Via headers (or application-specific equivalent). I > do not know whether the current requirement/architecture > drafts consider callout server a "proxy hop" on the > application protocol path. They probably should. > I think so, this should help in tracing/notification. abbie SNIP abbie ------_=_NextPart_001_01C2D9BD.FADD87C6 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES protocol, pre-draft

see inline,

abbie


> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Wednesday, February 19, 2003 3:37 = PM
> To: ietf-openproxy@imc.org
> Subject: Re: OPES protocol, pre-draft
>
>
>
>
> On Wed, 19 Feb 2003, jfcm wrote:
>
> > Let assume that www.w3c.com says to = redirect calls to
> www.w3.org and
> > that www.w3.org says to redirect calls to = www.w3c.com. What happens?
>
> A loop detection code at the second hop would = see its own
> Via: header already in the HTTP headers of the = request and
> would not forward the request further, = responding with an
> error. This is how redirections (and loop = detection) are
> supposed to work today.
>
yes, in theory.

> This issue is not directly related to OPES = except that OPES
> callout server might participate in loop = detection by adding
> its own Via headers (or application-specific = equivalent). I
> do not know whether the current = requirement/architecture
> drafts consider callout server a "proxy = hop" on the
> application protocol path. They probably = should.
>


I think so, this should help in = tracing/notification.

abbie

SNIP


abbie

------_=_NextPart_001_01C2D9BD.FADD87C6-- From owner-ietf-openproxy@mail.imc.org Fri Feb 21 11:46:37 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA06372 for ; Fri, 21 Feb 2003 11:46:36 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LGHcn10176 for ietf-openproxy-bks; Fri, 21 Feb 2003 08:17:38 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LGHad10172 for ; Fri, 21 Feb 2003 08:17:36 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1LGHbeM099681 for ; Fri, 21 Feb 2003 09:17:37 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1LGHbLM099680; Fri, 21 Feb 2003 09:17:37 -0700 (MST) (envelope-from rousskov) Date: Fri, 21 Feb 2003 09:17:37 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C120@hermes.webwasher.com> Message-ID: References: <72992B39BBD9294BB636A960E89AE02E50C120@hermes.webwasher.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Fri, 21 Feb 2003, Martin Stecher wrote: > ICAP/1.0 is already pipelined, I think. Messages are sent in chunked > transfer encoding. ICAP clients send chunk N before waiting for the > modified chunk N-1 to return. You are right, of course. I should have mentioned that ICAP does use pipelining. IIRC, ICAP pipes start only after message headers and do not allow any non-data messages to be present in-between (which forces ICAP to use a chunk-extension hack to indicate end-of-data). Another ICAP pipes disadvantage is that they cannot eliminate callout-to-processor data flow when the callout processor did not modify the data. The proposed protocol allows for this important common-case optimization using "data-asis" messages. Finally, ICAP pipes place ICAP connections in direct dependence on HTTP connections. If I have to process 5 HTTP messages _concurrently_, I have to open 5 ICAP connections. With the proposed approach, I can use 1, 2, 3, 4, or 5 OPES connections. This allows for possibly important performance optimizations on high-traffic servers. These limitations of ICAP model apparently come from it being an extension to HTTP (rather than being implemented on top of HTTP). OPES pipes are not an optimization add-on but a design feature. It remains to be seen if the proposed protocol is so much better that it actually makes sense to phase out a working/deployed ICAP protocol. Alex. From owner-ietf-openproxy@mail.imc.org Fri Feb 21 12:19:15 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA07525 for ; Fri, 21 Feb 2003 12:19:13 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LGsnL11154 for ietf-openproxy-bks; Fri, 21 Feb 2003 08:54:49 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LGsld11150 for ; Fri, 21 Feb 2003 08:54:47 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1LGsneM000595 for ; Fri, 21 Feb 2003 09:54:49 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1LGsnOX000594; Fri, 21 Feb 2003 09:54:49 -0700 (MST) (envelope-from rousskov) Date: Fri, 21 Feb 2003 09:54:49 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: Re: OPES protocol, additional message needed? In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C121@hermes.webwasher.com> Message-ID: References: <72992B39BBD9294BB636A960E89AE02E50C121@hermes.webwasher.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Martin, I agree that a message type is missing from the proposed protocol. While your motivation is perfectly valid, there is an even more general reasoning that leads to the same conclusion: The protocol relies on timeouts to resolve deadlocks from broken implementations and other situations where processing may get stuck. A timeout trigger would be reset if there is any activity related to a given transaction. If data stops flowing (because of a valid delay like loading a virus archive), there must be a way to create a data-less activity: "everything is normal, I am just slow, hang on". I suggest adding a still-here message: which can be used by both OPES processor and callout server, as needed. The message does not affect the state machine except it may reset the corresponding timeout trigger at the recipient side. If the sender uses no IDs, the message can be used to keep idle OPES connection alive and for similar keep-alive purposes unrelated to any particular transaction. If xid is used, all OPES-trouble-detection timeouts for that transaction should be reset. If amid is used, all trouble-detection timeouts for that application message should be reset (but other application messages may still timeout, depending on the state and protocol). Would keep-alive be a better name? I prefer still-here because it does not imply a specific action. It is up to the other side what to do when a still-here message is received. This blends well with OPES protocol design by keeping processor and callout server states as independent as possible. We probably do not care about specific names at this point though. I hesitate adding "change-probability" and "progress" fields. I am not convinced that they are the best way to implement what you want. change-probability field: if the primary purpose is to fight latency, we should address that directly. Latency does not have to be related to pending _changes_: - Download progress indication: Always a good idea if application protocol supports it. Callout server is a poor judge of the progress though because it is just a hop in the pipeline. There may be more callout servers, for example. OPES processor is in a better position to estimate progress (based on past performance). User Agent is in an even better position (and most support this feature!). - Data trickling: same as download progress indication; if application protocol supports sending empty messages, they can be sent. Knowing change-probability does not help with that. If callout server can trickle real data, it should do it anyway. - LateClearance: seems like this should ALWAYS be used (if possible) for content that MAY have a virus because change-probability for such content is always positive at the beginning and because LateClearance overhead can be made very low. progress field: the semantics seems to be covered by the still-here message itself. "I am still here (doing something, naturally), hang on if you can!" Am I downplaying the importance of change-probability and progress fields? Do you expect implementations hack them in as OPES extensions if we do not define them? Are there corresponding ICAP extensions already? On a related note, would it be of any benefit to add a flag to data-have message from the callout server indicating that the corresponding data has been modified by the callout server? Or should we address that when we start talking about authentication/privacy/etc. support? Thank you, Alex. On Fri, 21 Feb 2003, Martin Stecher wrote: > > There is one common scenario with virus filtering in a callout server > that I wrote about some time ago. > > Do you think an additional message type could help and is worth here? > > Scenario: > A virus filter often cannot send chunks of data back to the OPES > processor before it has the complete data. Only then it can start > the scanning process. > On the other hand it modifies data only in rare situations. > Most results are "no virus found, use original message unmodified". > Only if virus was found, it modifies (repair) the data or replaces > it in parts or completly by an error message. > > So a possible message from callout server to OPES processor could be > > > > It informs the OEPS processor that the callout server is working on > this although it does not reply data now. If "change-probability" is > low, the OPES processor could use techniques to fight the raising > latency time, > for example: > - some kind of download progress indication, > - "data trickling" or > - the proposed LateClearance content encoding > (see draft-stecher-lclr-encoding-00.txt) > > The "progress" parameter can inform that the callout processor needs > even more time although all data was already received (e.g. an archive > needs to be extracted first which is a currently ongoing lengthy > process) > > > Thanks > Martin From owner-ietf-openproxy@mail.imc.org Fri Feb 21 12:36:37 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA08130 for ; Fri, 21 Feb 2003 12:36:36 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LHGLF11790 for ietf-openproxy-bks; Fri, 21 Feb 2003 09:16:21 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1LHGJd11786 for ; Fri, 21 Feb 2003 09:16:19 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id D6AC2IFP; 21 Feb 2003 18:16:19 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: OPES protocol, pre-draft X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Fri, 21 Feb 2003 18:13:13 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C123@hermes.webwasher.com> Thread-Topic: OPES protocol, pre-draft Thread-Index: AcLZxetC4fghWBPjTXuZ2+NbXoXofQAA7JCA From: "Martin Stecher" To: "Alex Rousskov" , Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1LHGKd11787 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit > > ICAP/1.0 is already pipelined, I think. Messages are sent in chunked > > transfer encoding. ICAP clients send chunk N before waiting for the > > modified chunk N-1 to return. > > You are right, of course. I should have mentioned that ICAP does use > pipelining. IIRC, ICAP pipes start only after message headers and do > not allow any non-data messages to be present in-between (which forces > ICAP to use a chunk-extension hack to indicate end-of-data). > > Another ICAP pipes disadvantage is that they cannot eliminate > callout-to-processor data flow when the callout processor did not > modify the data. The proposed protocol allows for this important > common-case optimization using "data-asis" messages. ICAP's 204 response is something like this, isn't it? Similar to your proposal, the ICAP client needs to advertise that it keeps a copy of the data (by adding the Allow: 204 ICAP header) and the ICAP server can then respond with "ICAP/1.0 204 Not modified" not only after the preview but after it has seen more or all of the HTTP data. I see that the OPES pipe proposal goes further, it is really a next generation protocol. > > Finally, ICAP pipes place ICAP connections in direct dependence on > HTTP connections. If I have to process 5 HTTP messages _concurrently_, > I have to open 5 ICAP connections. With the proposed approach, I can > use 1, 2, 3, 4, or 5 OPES connections. This allows for possibly > important performance optimizations on high-traffic servers. Fully agree. > > These limitations of ICAP model apparently come from it being an > extension to HTTP (rather than being implemented on top of HTTP). > OPES pipes are not an optimization add-on but a design feature. > > It remains to be seen if the proposed protocol is so much better that > it actually makes sense to phase out a working/deployed ICAP protocol. Yes, let's first get further with a proposal for a best-we-can-think-of- OPES-protocol. You wrote in a message on Wednesday: > I expect ICAP proponents defend ICAP on this list. There are probably not too many stronger supporters of ICAP than I am. And I will proudly defend ICAP. But this does not mean that we cannot work on something new and better. It took nearly two years from a stable protocol draft to an almost RFC status and about 18 month before we could convince a significant number of vendors to support ICAP in their products. The differences between ICAP/0.9, ICAP/0.95 and ICAP/1.0 are very noticable. So, I do not see any argument why we cannot create a next generation protocol that becomes at least as successful as ICAP/1.0. Yes, it will take some time and it has to be much better. But this is the fun part, isn't it? I like to talk about ICAP TNG ("the next generation" for all non-Trekkies) as a working title because I think that the protocol's success can be maximized if it really looks like a successor of ICAP/1.0 and is maybe even named ICAP/2.0. As far as I see the current proposal could still work out in this direction (and also in something very different, I know). This is of course not in the focus of this WG and I understand that some of you do not like this direction. It is just my personal motivation (sorry for bothering you with this ;-) Martin From owner-ietf-openproxy@mail.imc.org Fri Feb 21 12:57:21 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA09130 for ; Fri, 21 Feb 2003 12:57:20 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LHYse12868 for ietf-openproxy-bks; Fri, 21 Feb 2003 09:34:54 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1LHYqd12864 for ; Fri, 21 Feb 2003 09:34:52 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id UP582U41; 21 Feb 2003 18:34:52 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: RE: OPES protocol, additional message needed? X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Fri, 21 Feb 2003 18:31:47 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C124@hermes.webwasher.com> Thread-Topic: OPES protocol, additional message needed? Thread-Index: AcLZyyqnjpxi4zU0Tyybik1wTHN+2AAApT/g From: "Martin Stecher" To: "Alex Rousskov" , "OPES WG (E-mail)" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1LHYrd12865 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit Hi Alex, >[...] > > Am I downplaying the importance of change-probability and progress > fields? Do you expect implementations hack them in as OPES extensions > if we do not define them? Are there corresponding ICAP extensions > already? >[...] There is no message like this in ICAP today and I am not aware of an extension for change-probability. The result is that ICAP servers start to implement the donwload progress indication / data trickling features and as you pointed out this belongs in the OPES processor. But the OPES processor needs to have some input to determine whether and what latency fighting method to use. The overall question is: If I (the OPES processor) start to forward original application message data (by data trickling or LateClearance content encoding), how likely is it that I will get into deep trouble when the callout server sends modified data? It can only know if the callout server has sent some information about this likelihood. And regarding progress: Let's assume there is a way to forward progress indication to the consumer, then the OPES processor can better work with a numeric value (e.g. "still 10 seconds to go" or "30% done") rather then only "the callout server is still working on it". I know that there was some work to get this feature into an ICAP extension but it has not yet been implemented. Regards Martin From owner-ietf-openproxy@mail.imc.org Fri Feb 21 13:04:43 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA09366 for ; Fri, 21 Feb 2003 13:04:42 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LHgRO13601 for ietf-openproxy-bks; Fri, 21 Feb 2003 09:42:27 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LHgQd13597 for ; Fri, 21 Feb 2003 09:42:26 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1LHgSeM001758 for ; Fri, 21 Feb 2003 10:42:28 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1LHgRvd001757; Fri, 21 Feb 2003 10:42:28 -0700 (MST) (envelope-from rousskov) Date: Fri, 21 Feb 2003 10:42:27 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C123@hermes.webwasher.com> Message-ID: References: <72992B39BBD9294BB636A960E89AE02E50C123@hermes.webwasher.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Fri, 21 Feb 2003, Martin Stecher wrote: > > Another ICAP pipes disadvantage is that they cannot eliminate > > callout-to-processor data flow when the callout processor did not > > modify the data. The proposed protocol allows for this important > > common-case optimization using "data-asis" messages. > > ICAP's 204 response is something like this, isn't it? Similar to > your proposal, the ICAP client needs to advertise that it keeps a > copy of the data (by adding the Allow: 204 ICAP header) and the ICAP > server can then respond with "ICAP/1.0 204 Not modified" not only > after the preview but after it has seen more or all of the HTTP > data. Yes, 204 Not modified works similarly but can only be applied to unmodified message tails (including entire messages) whether OPES "asis" mechanism aplies to individual data chunks in the pipeline. 204 Not modified is a one-time all-or-nothing decision. Same idea (asis feature was inspired by ICAP of course), different scope. Asis feature can be used to reduce data transfer from callout server to processor to the necessary minimum even if callout server modifies the last chunk of the message. > So, I do not see any argument why we cannot create a next generation > protocol that becomes at least as successful as ICAP/1.0. Yes, it > will take some time and it has to be much better. But this is the > fun part, isn't it? Exactly. > I like to talk about ICAP TNG ("the next generation" for all > non-Trekkies) as a working title because I think that the protocol's > success can be maximized if it really looks like a successor of > ICAP/1.0 and is maybe even named ICAP/2.0. As far as I see the > current proposal could still work out in this direction (and also in > something very different, I know). I agree that calling OPES protocol ICAP may help OPES acceptance/deployment for non-technical reasons. To save time and flames, I suggest to avoid the naming issue until OPES protocol is shaped and the consensus is, in fact, that it is better than ICAP/1.0. Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Fri Feb 21 13:40:30 2003 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA10444 for ; Fri, 21 Feb 2003 13:40:30 -0500 (EST) Received: from above.proper.com (mail.proper.com [208.184.76.45]) by ietf-mx.ietf.org (8.11.6+Sun/8.11.6) with ESMTP id h1LIQk800406 for ; Fri, 21 Feb 2003 13:26:47 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LIHe515141 for ietf-openproxy-bks; Fri, 21 Feb 2003 10:17:40 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LIHdd15137 for ; Fri, 21 Feb 2003 10:17:39 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1LIHSa17712; Fri, 21 Feb 2003 13:17:28 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6WXR0>; Fri, 21 Feb 2003 13:17:29 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404F1E076@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft Date: Fri, 21 Feb 2003 13:17:28 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D9D5.7DE200E0" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D9D5.7DE200E0 Content-Type: text/plain hi, see inside please. abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Friday, February 21, 2003 12:42 PM > To: ietf-openproxy@imc.org > Subject: RE: OPES protocol, pre-draft > > I agree that calling OPES protocol ICAP may help OPES > acceptance/deployment for non-technical reasons. To save time > and flames, I suggest to avoid the naming issue until OPES > protocol is shaped and the consensus is, in fact, that it is > better than ICAP/1.0. > > Thank you, > > Alex. Let us be carefull here. So far my main concern with the proposed protocol is that it looks a lot like ICAP. The main scenarioes that are solved so far are basically packaging (or passing )a request to the callout server, with all the technicalities. We need to start thinking about scenarios that involves the user preferences, that include the choice of obtaining a service from more than one source etc.. It is far too early to start even thinking about naming the OPES callout protocol ICAP xxxx. abbie ------_=_NextPart_001_01C2D9D5.7DE200E0 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES protocol, pre-draft

hi,

see inside please.

abbie


> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Friday, February 21, 2003 12:42 PM
> To: ietf-openproxy@imc.org
> Subject: RE: OPES protocol, pre-draft
>

> I agree that calling OPES protocol ICAP may help = OPES
> acceptance/deployment for non-technical = reasons. To save time
> and flames, I suggest to avoid the naming issue = until OPES
> protocol is shaped and the consensus is, in = fact, that it is
> better than ICAP/1.0.
>
> Thank you,
>
> Alex.

Let us be carefull here. So far my main concern with = the proposed protocol is that it looks a lot like ICAP.  The main = scenarioes that are solved so far are basically packaging (or passing = )a request to the callout server, with all the = technicalities.

We need to start thinking about scenarios that = involves the user preferences, that include the choice of obtaining a = service from more than one source etc..

It is far too early to start even thinking about = naming the OPES callout protocol ICAP xxxx.

abbie








------_=_NextPart_001_01C2D9D5.7DE200E0-- From owner-ietf-openproxy@mail.imc.org Fri Feb 21 13:48:19 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA10800 for ; Fri, 21 Feb 2003 13:48:18 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LINEi15579 for ietf-openproxy-bks; Fri, 21 Feb 2003 10:23:14 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LINCd15572 for ; Fri, 21 Feb 2003 10:23:12 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1LINEeM002696 for ; Fri, 21 Feb 2003 11:23:14 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1LINEQJ002695; Fri, 21 Feb 2003 11:23:14 -0700 (MST) (envelope-from rousskov) Date: Fri, 21 Feb 2003 11:23:14 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft In-Reply-To: <87609AFB433BD5118D5E0002A52CD75404F1E076@zcard0k6.ca.nortel.com> Message-ID: References: <87609AFB433BD5118D5E0002A52CD75404F1E076@zcard0k6.ca.nortel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Fri, 21 Feb 2003, Abbie Barbir wrote: > Let us be carefull here. So far my main concern with the proposed > protocol is that it looks a lot like ICAP. The main scenarioes that > are solved so far are basically packaging (or passing )a request to > the callout server, with all the technicalities. > > We need to start thinking about scenarios that involves the user > preferences, that include the choice of obtaining a service from > more than one source etc.. > > It is far too early to start even thinking about naming the OPES > callout protocol ICAP xxxx. I agree with all of the above. The major challenges are still ahead. Message passing is just one (most performance-sensitive, but not the most difficult) part of the protocol. Alex. From owner-ietf-openproxy@mail.imc.org Fri Feb 21 13:51:31 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA10900 for ; Fri, 21 Feb 2003 13:51:29 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LITWt16350 for ietf-openproxy-bks; Fri, 21 Feb 2003 10:29:32 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LITVd16343 for ; Fri, 21 Feb 2003 10:29:31 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1LITXeM002822 for ; Fri, 21 Feb 2003 11:29:33 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1LITXrm002821; Fri, 21 Feb 2003 11:29:33 -0700 (MST) (envelope-from rousskov) Date: Fri, 21 Feb 2003 11:29:33 -0700 (MST) From: Alex Rousskov To: "OPES WG (E-mail)" Subject: RE: OPES protocol, additional message needed? In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C124@hermes.webwasher.com> Message-ID: References: <72992B39BBD9294BB636A960E89AE02E50C124@hermes.webwasher.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Martin, I agree 50%: Content modification is a primary OPES function. Probability of a modification can be expressed in simple terms. Moreover it may affect how the application message is handled (e.g., encoding or dropping if things go wrong). We probably should support this in the core protocol. Note that LateClearance has to be decided before any headers are sent; it must be possible to attach the hint to the consumer-start OPES message. I will try to make necessary modifications. As for progress, my concern is that it is impossible to have a simple machine-friendly format that describes "progress". Simple format cannot express both "still 10 seconds to go" and "30% done", and there are many other variations that some applications may want. Moreover, "still 10 seconds to go" and "30% done" can be very wrong if they are sent to the client without adjustments (OPES may not know of all other hops on the way). Neither HTTP nor SMTP intermediaries have any sane way of reporting this kind of progress back to the client. I am afraid of adding pages of complicated features to the protocol that are not going to be used much. To me, it smells like a good candidate for a protocol extension (we still need to built in mechanism for those) not a core feature. An extension may still be very well documented and widely supported, of course. Do you think that progress indication must be a core OPES feature? Thank you, Alex. On Fri, 21 Feb 2003, Martin Stecher wrote: > There is no message like this in ICAP today and I am not aware of an > extension for change-probability. The result is that ICAP servers > start to implement the donwload progress indication / data trickling > features and as you pointed out this belongs in the OPES processor. > > But the OPES processor needs to have some input to determine whether > and what latency fighting method to use. The overall question is: If > I (the OPES processor) start to forward original application message > data (by data trickling or LateClearance content encoding), how > likely is it that I will get into deep trouble when the callout > server sends modified data? It can only know if the callout server > has sent some information about this likelihood. > > And regarding progress: Let's assume there is a way to forward > progress indication to the consumer, then the OPES processor can > better work with a numeric value (e.g. "still 10 seconds to go" or > "30% done") rather then only "the callout server is still working on > it". > > I know that there was some work to get this feature into an ICAP > extension but it has not yet been implemented. > > > Regards > Martin > > From owner-ietf-openproxy@mail.imc.org Fri Feb 21 13:59:45 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA11171 for ; Fri, 21 Feb 2003 13:59:44 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LIc9Z17515 for ietf-openproxy-bks; Fri, 21 Feb 2003 10:38:09 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1LIc7d17505 for ; Fri, 21 Feb 2003 10:38:07 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id 9P29Y0CO; 21 Feb 2003 19:38:08 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: AW: OPES protocol, pre-draft X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Fri, 21 Feb 2003 19:35:02 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E8FD225@hermes.webwasher.com> Thread-Topic: OPES protocol, pre-draft Thread-Index: AcLZ11LRWM7kh3RhR8y6RWlj4yXkWgAAP2aA From: "Martin Stecher" To: Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1LIc8d17508 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit You are right. Martin > -----Ursprüngliche Nachricht----- > Von: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Gesendet: Freitag, 21. Februar 2003 19:23 > An: ietf-openproxy@imc.org > Betreff: RE: OPES protocol, pre-draft > > > > On Fri, 21 Feb 2003, Abbie Barbir wrote: > > > Let us be carefull here. So far my main concern with the proposed > > protocol is that it looks a lot like ICAP. The main scenarioes that > > are solved so far are basically packaging (or passing )a request to > > the callout server, with all the technicalities. > > > > We need to start thinking about scenarios that involves the user > > preferences, that include the choice of obtaining a service from > > more than one source etc.. > > > > It is far too early to start even thinking about naming the OPES > > callout protocol ICAP xxxx. > > I agree with all of the above. The major challenges are still ahead. > Message passing is just one (most performance-sensitive, but not the > most difficult) part of the protocol. > > Alex. > From owner-ietf-openproxy@mail.imc.org Fri Feb 21 14:15:07 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA11690 for ; Fri, 21 Feb 2003 14:15:06 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LIp8G19660 for ietf-openproxy-bks; Fri, 21 Feb 2003 10:51:08 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1LIp6d19656 for ; Fri, 21 Feb 2003 10:51:06 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id B7IU9IR2; 21 Feb 2003 19:51:07 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Subject: AW: OPES protocol, additional message needed? X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Fri, 21 Feb 2003 19:48:01 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E8FD226@hermes.webwasher.com> Thread-Topic: OPES protocol, additional message needed? Thread-Index: AcLZ2BXKGWeXZJP+TGGLCtPNE+ifKAAATyag From: "Martin Stecher" To: "Alex Rousskov" , "OPES WG (E-mail)" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1LIp7d19657 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit Alex, the progress parameter is not a must have core feature. As you described it will not be used in many cases. If there is a way to extend this later, fine. The probability parameter should go into the core. Do we now agree 100%? ;-) Martin > -----Ursprüngliche Nachricht----- > Von: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Gesendet: Freitag, 21. Februar 2003 19:30 > An: OPES WG (E-mail) > Betreff: RE: OPES protocol, additional message needed? > > > > Martin, > > I agree 50%: Content modification is a primary OPES function. > Probability of a modification can be expressed in simple terms. > Moreover it may affect how the application message is handled (e.g., > encoding or dropping if things go wrong). We probably should support > this in the core protocol. > > Note that LateClearance has to be decided before any headers > are sent; it must be possible to attach the hint to the consumer-start > OPES message. I will try to make necessary modifications. > > > > As for progress, my concern is that it is impossible to have a > simple machine-friendly format that describes "progress". Simple > format cannot express both "still 10 seconds to go" and "30% done", > and there are many other variations that some applications may want. > Moreover, "still 10 seconds to go" and "30% done" can be very wrong if > they are sent to the client without adjustments (OPES may not know of > all other hops on the way). > > Neither HTTP nor SMTP intermediaries have any sane way of > reporting this kind of progress back to the client. I am afraid of > adding pages of complicated features to the protocol that are not > going to be used much. To me, it smells like a good candidate for a > protocol extension (we still need to built in mechanism for those) not > a core feature. An extension may still be very well documented and > widely supported, of course. Do you think that progress indication > must be a core OPES feature? > > Thank you, > > Alex. > > > On Fri, 21 Feb 2003, Martin Stecher wrote: > > > There is no message like this in ICAP today and I am not aware of an > > extension for change-probability. The result is that ICAP servers > > start to implement the donwload progress indication / data trickling > > features and as you pointed out this belongs in the OPES processor. > > > > But the OPES processor needs to have some input to determine whether > > and what latency fighting method to use. The overall question is: If > > I (the OPES processor) start to forward original application message > > data (by data trickling or LateClearance content encoding), how > > likely is it that I will get into deep trouble when the callout > > server sends modified data? It can only know if the callout server > > has sent some information about this likelihood. > > > > And regarding progress: Let's assume there is a way to forward > > progress indication to the consumer, then the OPES processor can > > better work with a numeric value (e.g. "still 10 seconds to go" or > > "30% done") rather then only "the callout server is still working on > > it". > > > > I know that there was some work to get this feature into an ICAP > > extension but it has not yet been implemented. > > > > > > Regards > > Martin > > > > > From owner-ietf-openproxy@mail.imc.org Fri Feb 21 14:22:34 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA11945 for ; Fri, 21 Feb 2003 14:22:32 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LIxUt20004 for ietf-openproxy-bks; Fri, 21 Feb 2003 10:59:30 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LIxTd20000 for ; Fri, 21 Feb 2003 10:59:29 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1LIxVeM003521 for ; Fri, 21 Feb 2003 11:59:31 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1LIxV1F003520; Fri, 21 Feb 2003 11:59:31 -0700 (MST) (envelope-from rousskov) Date: Fri, 21 Feb 2003 11:59:30 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft In-Reply-To: <5.2.0.9.0.20030221150123.02af8940@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030221150123.02af8940@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Fri, 21 Feb 2003, jfcm wrote: > Sorry, I though I had sent back to Hillarie and the list - I hate > these no reply to the list - quite confusing to be forced to talk > privately on a public place. Better to be forced to talk privately than publicly! But you should not blame the mailing list organization -- blame your mail agent instead, for not auto-filling both TO and CC headers :-). > What I see from the mails is: > > 1. there is a clear understanding of what OPES are and how they > could work in using SOAP, XML etc. CPU and line bandwidth > consuming standard solutions. This is very interesting as it is > a good consistent Internet response style with what is going > around. It seems an exact, clear response to the IAB desire. > > 2. there is a smarter, pipe oriented, low cpu load approach with > user interaction which corresponds to what I look for and which > is actually like a network operating system. Perhaps it is just me, but I am still not quite sure what ONES is or should be. Could you please give two very specific, real-world ONES examples? Clearly, if there is a smarter, lower-load architecture we should at least consider it! Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Fri Feb 21 14:27:14 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA12067 for ; Fri, 21 Feb 2003 14:27:12 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LJ1tF20093 for ietf-openproxy-bks; Fri, 21 Feb 2003 11:01:55 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LJ1sd20088 for ; Fri, 21 Feb 2003 11:01:54 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1LJ1ueM004573 for ; Fri, 21 Feb 2003 12:01:56 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1LJ1u9p004572; Fri, 21 Feb 2003 12:01:56 -0700 (MST) (envelope-from rousskov) Date: Fri, 21 Feb 2003 12:01:56 -0700 (MST) From: Alex Rousskov To: "OPES WG (E-mail)" Subject: Re: OPES protocol, additional message needed? In-Reply-To: <72992B39BBD9294BB636A960E89AE02E8FD226@hermes.webwasher.com> Message-ID: References: <72992B39BBD9294BB636A960E89AE02E8FD226@hermes.webwasher.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Fri, 21 Feb 2003, Martin Stecher wrote: > the progress parameter is not a must have core feature. As you > described it will not be used in many cases. If there is a way to > extend this later, fine. > > The probability parameter should go into the core. > > Do we now agree 100%? ;-) Yes. I will make necessary modifications. Alex. From owner-ietf-openproxy@mail.imc.org Fri Feb 21 15:01:07 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA12893 for ; Fri, 21 Feb 2003 15:01:06 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LJYVx21098 for ietf-openproxy-bks; Fri, 21 Feb 2003 11:34:31 -0800 (PST) Received: from zcars0m9.nortelnetworks.com (zcars0m9.nortelnetworks.com [47.129.242.157]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LJYUd21094 for ; Fri, 21 Feb 2003 11:34:31 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars0m9.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1LJYKa05006; Fri, 21 Feb 2003 14:34:20 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6WZMM>; Fri, 21 Feb 2003 14:34:20 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404F1E198@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft Date: Fri, 21 Feb 2003 14:34:17 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2D9E0.394FF6DE" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2D9E0.394FF6DE Content-Type: text/plain Hi, Thanks abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Friday, February 21, 2003 1:23 PM > To: ietf-openproxy@imc.org > Subject: RE: OPES protocol, pre-draft > > > > On Fri, 21 Feb 2003, Abbie Barbir wrote: > > > Let us be carefull here. So far my main concern with the proposed > > protocol is that it looks a lot like ICAP. The main > scenarioes that > > are solved so far are basically packaging (or passing )a request to > > the callout server, with all the technicalities. > > > > We need to start thinking about scenarios that involves the user > > preferences, that include the choice of obtaining a service > from more > > than one source etc.. > > > > It is far too early to start even thinking about naming the OPES > > callout protocol ICAP xxxx. > > I agree with all of the above. The major challenges are still > ahead. Message passing is just one (most > performance-sensitive, but not the most difficult) part of > the protocol. > > Alex. > ------_=_NextPart_001_01C2D9E0.394FF6DE Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES protocol, pre-draft

Hi,
Thanks

abbie

> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Friday, February 21, 2003 1:23 PM
> To: ietf-openproxy@imc.org
> Subject: RE: OPES protocol, pre-draft
>
>
>
> On Fri, 21 Feb 2003, Abbie Barbir wrote:
>
> > Let us be carefull here. So far my main = concern with the proposed
> > protocol is that it looks a lot like = ICAP.  The main
> scenarioes that
> > are solved so far are basically packaging = (or passing )a request to
> > the callout server, with all the = technicalities.
> >
> > We need to start thinking about scenarios = that involves the user
> > preferences, that include the choice of = obtaining a service
> from more
> > than one source etc..
> >
> > It is far too early to start even thinking = about naming the OPES
> > callout protocol ICAP xxxx.
>
> I agree with all of the above. The major = challenges are still
> ahead. Message passing is just one (most =
> performance-sensitive, but not the most = difficult) part of
> the protocol.
>
> Alex.
>

------_=_NextPart_001_01C2D9E0.394FF6DE-- From owner-ietf-openproxy@mail.imc.org Fri Feb 21 17:00:40 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA16407 for ; Fri, 21 Feb 2003 17:00:36 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1LLWx426091 for ietf-openproxy-bks; Fri, 21 Feb 2003 13:32:59 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1LLWwd26087 for ; Fri, 21 Feb 2003 13:32:58 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1LLX1eM008062 for ; Fri, 21 Feb 2003 14:33:01 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1LLX1en008061; Fri, 21 Feb 2003 14:33:01 -0700 (MST) (envelope-from rousskov) Date: Fri, 21 Feb 2003 14:33:01 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: OPES protocol, pre-draft 01 Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: There were many great suggestions on how to improve the protocol pre-draft. Please find an updated document below. Here is the change log: - replaced "bid" with "amid" attribute and fixed amid definition due to many compliants and confusions - replaced OPES server with callout server (Oskar Batuner) - added "sizep", an optional message size prediction attribute (Hilarie Orman) - added "modp", an optional modification prediction attribute (Martin Stecher) - added messages (Martin Stecher) - simplified document structure; removed many general remarks and moved essential ones into a short Introduction section; polished text - added TODO section Comments and help with the TODO list are requested. Thank you, Alex. ------------------------ Table of Contents: 0. Introduction. 1. Message properties 2. Message types 3. Examples 4. Transport connections 5. Synchronization and error handling 6. About this document 7. TODO 0. Introduction draft-ietf-opes-protocol-reqs-03.txt defines the following information flow: data provider --(original application message)--> -- [ OPES magic ] --> --(produced application messages)--> data consumers The original and "produced" (forwarded) messages together form an application protocol transaction. Note that there may be more than one produced application message resulting from a single original message. When application protocol involves a request-response sequence (e.g., HTTP), we treat it as two related OPES transactions: request transaction and response transaction. OPES processor and callout server exchange messages. The exchange is bidirectional. There is no clear client or server role. There is no clear request/response message category either. OPES messages manipulate the state of these four buffers/connections and associated meta-information: - data producer (incoming) buffer at the OPES processor - data producer (incoming) buffer at the callout server - data consumer (outgoing) buffer at the callout server - data consumer (outgoing) buffer at the OPES processor The design prevents buffer overflows and allows to discard buffered content as soon as possible. Note that we rely on OPES transport protocol to be both reliable and to stop sending us more data (eventually) if we stop reading it. TCP has both properties. [ Note: The XML-like syntax for describing protocol parts does NOT imply that OPES messages should be implemented using XML. Text or binary encodings can be used; the encoding decision is out of scope of this document. ] 1. Message properties Many OPES messages share the following properties. xid -- Application transaction identifier (Xaction ID) Uniquely identifies an application transaction among all OPES agents that may see this ID. amid -- Application Message IDentifier Uniquely identifies an application message within an application transaction. Amid can be interpreted in an application transaction context only. Thus, either xid must be present whenever amid is used or amid must uniquely identify application transaction as well (e.g., by containing xid). [ @@@ we should decide one way or the other ] source -- Information about the data provider (i.e., the source of the application message). For messages originated from the OPES processor, the source describes the original data provider. For messages originated from the callout server, the source describes what provider information should be presented to the data consumer; callout server may need to change how the original information looks to the other application side. destinations -- One or more destinations. Depending on the application, OPES processor may need to check that all original destinations have been covered by callout server. destination -- Information about the data consumer (i.e, the destination of the application message). For messages originated from the OPES processor, destination describes the consumer as intended by the producer. For messages originated from the callout server, the destination is the data consumer that should be used by the OPES processor; callout server may need to change the intended recipient. services -- One or more services. There will be a way to indicate desired order of service application, possibly including concurrent applications at the callout server data size -- Specific data size in octets OR a special token meaning "all" or "maximum". The all-token may only be used when requesting data, never when sending it. data offset -- non-negative number of octets relative to the beginning of the application message sizep -- size prediction An integer value of at least zero. Size-prediction property carries remaining application message size prediction, in octets. The value includes data in the current message, if any. This property can be used in any message with amid property. This is a prediction, not a fact. modp -- modification probability prediction with An integer value from 0 to 100, indicating the probability (0 = will probably never happen, 100 = probably imminent) that some produced data following the prediction (including data in the current message, if any) will differ from the original data. A reading of 100 does not imply that the current (or any!) message data has been modified. This is a prediction, not a fact. This property can be present in any callout server message with amid property. Absence of the property means absence of a [new] prediction, not that there will be no modifications. Note that prediction is persistent for the given amid unless overwritten by a different value of modp in a later message. [ @@@ if OPES can change meta-info like destination address, should that be included in modification semantics? ] reason -- This should probably be a numeric status code with an optional information string. In examples, we will use just strings for now. 2. Message types An OPES processor may send the following messages to the callout server. Informs callout server about a new application transaction. This message should probably identify OPES service(s) requested for this transaction and other transaction-global info unrelated to data buffering, sources, or destinations. Informs callout server about a new message from the data producer. Amid can probably be set to xid unless we expect to handle protocols that may merge messages before forwarding them. Sends [a portion of] application message from the data producer buffer to the callout server. If "copied" flag is set, the callout server may assume that the corresponding data is buffered at the processor and may refer to it using messages described below. Copying commitment must last until the corresponding message or event. Notifies callout server that there will be no more data for this transaction (coming from the OPES processor) UNLESS callout server explicitly asks for it using message described below. This message may be used if OPES processor suspects that callout server is not interested in the data and, hence, there is no reason to send it by default (e.g., a response content type indicates that it is unlikely to have a virus but only callout server can know for sure). Notifies callout server that there will be no more data for this transaction (coming from the OPES processor) Notifies callout server that there will be no more messages for this amid (coming from the OPES processor) Notifies callout server that there will be no more messages for this transaction (coming from the OPES processor) A callout server may send the following messages to the OPES processor. Informs OPES processor that callout server may want to send data from source to destination(s). There may be other messages (amids) associated with the same transaction (xid). Xid comes from the corresponding xaction-start message send by the OPES processor. Tells OPES processor to send the attached data to the data consumer. Tells OPES processor to use processor's own copy of the specified data to send to the data consumer. This message can only specify data fragments previously marked with "copied" flag in a message from OPES processor. Tells OPES processor that the callout server will never send data-as-is message for the specified data range. This message can only specify data fragments previously marked with the "copied" flag in a message from OPES processor. This message amid must match the (producer) message amid, not the consumer amid. This optional message may help OPES processor to free its resources. Tells OPES processor to send the specified data segment to the callout server (probably in response to data-pause message from the callout server). This message amid must match the corresponding producer amid, not the consumer amid. Notifies OPES processor that it should not send more data for this transaction until callout server explicitly asks for it using data-need message described above. This message amid must match the corresponding producer amid, not the consumer amid. Tells OPES processor that there will be no more data for this amid (coming from the callout server) Notifies callout server that there will be no more messages for this amid (coming from the callout server) Notifies callout server that there will be no more messages for this transaction (coming from the callout server) Note: There needs to be a way for callout server to tell OPES processor to terminate (or short-circuit) the forwarding of a message. This feature needs to be added to the protocol, but it should not change the overall design. One way to support this feature is for callout server to change the destination of the application message from consumer to producer (and change source to itself?). OPES processor or callout server may send the following messages. The messages tell recipient that the sender is working, working on xid, or working on amid, respectively. The sender may not be able to send any other message (yet), but wants to inform the recipient that it knows of recipient's (or xid, or amid) existence. The sender MAY send more specific messages later. 3. Examples Here is an example of (not) filtering an HTTP message based on HTTP headers: processor: processor: processor: processor: server: server: server: Note that xaction-end implies consumer-end implies data-end, and there is no reason for OPES processor to send a xaction-end message to server if the server already sent xaction-end message. The lines above are grouped about possible network I/O boundaries; thus, only two network data packets may be required to process a message if the callout server decides it does not care based on the headers. Here is an example of redirecting an HTTP request by changing its destination info and corresponding HTTP headers: processor: processor: processor: processor: server: server: server: Finally, here is an example of modifying the "middle" part of HTTP message body. The callout server switches the message encoding to chunked, to avoid buffering data to figure out new Content-Length. processor: processor: processor: processor: server: server: server: server: processor: server: processor: /* send modified chunk, tell processor to ignore the original */ server: server: processor: processor: server: server: Note that once the flow starts, there are no explicit synchronization points or waiting. The above message order is not the only one possible: most messages from the processor are not synchronized with most messages from the server. 4. Transport connections OPES transport connections would depend on the transport protocol (HTTP, BEEP, etc.). It is important to note that regardless of the transport protocol chosen, it is possible to multiplex messages from the OPES processor (or from the server) over several persistent connections. OPES messages do not depend on "connection" properties except for the basic requirement that order-dependent messages use the same transport connection, in the right order. 5. Synchronization and error handling. The protocol has very few explicit dependencies between messages. It is trivial to imagine a case where incorrect processor or server implementation would result in deadlocks or other bad states. All sorts of deadlocks are resolved using timeouts. If there is no progress with the transaction for an admin-configurable time, the transaction is aborted. Aborting at callout server side is easy: server: On the processor side, specific actions would depend on the protocol and state. For example, if no response bytes have been sent to an HTTP client yet, then an error response can be sent. It would be also possible, in some states, to eliminate OPES server from processing if it fails. Supporting this behavior would require having a copy of entire application messages even is callout server tells us it does not need a copy. The exact behavior must be admin-configurable. 6. About this document This document goal is to become a section in the future OPES protocol specs, after a lot of editing. The OPES milestone reads: "MAY 03 Initial protocol document for OPES services including their authorization, invocation, tracking, and enforcement of authorization". 7. TODO 1. Decide whether callout server can change application message source and destinations 2. Understand and support the following: "It should be possible to indicate that the transmitted data comes from several places in the amid. This allows the OPES processor to omit huge cookies and other junk; the response, by including this information, helps the process limit the state and parsing." 3. Document how one can write OPES extensions. Use "progress meter" as an example/motivation. $Id: protocol.txt,v 1.2 2003/02/21 21:25:14 rousskov Exp rousskov $ From owner-ietf-openproxy@mail.imc.org Sat Feb 22 15:54:19 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA15435 for ; Sat, 22 Feb 2003 15:54:18 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1MK7pC16109 for ietf-openproxy-bks; Sat, 22 Feb 2003 12:07:51 -0800 (PST) Received: from sccrmhc01.attbi.com (sccrmhc01.attbi.com [204.127.202.61]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1MK7od16105 for ; Sat, 22 Feb 2003 12:07:50 -0800 (PST) Received: from oskar3 (h00a0c5e5337d.ne.client2.attbi.com[24.218.184.133]) by sccrmhc01.attbi.com (sccrmhc01) with SMTP id <2003022220074700100rn0kde>; Sat, 22 Feb 2003 20:07:47 +0000 From: "Oskar Batuner" To: "Alex Rousskov" , "OPES WG" Subject: RE: OPES protocol, pre-draft 01 Date: Sat, 22 Feb 2003 15:08:21 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal In-Reply-To: Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit Hi Alex, Here are some comments: > 0. Introduction > > draft-ietf-opes-protocol-reqs-03.txt defines the following > information flow: > > data provider --(original application message)--> > -- [ OPES magic ] --> > --(produced application messages)--> data consumers > I think both protocol requirements and architecture support OPES processing in bi-directional message flow, both for data consumer requests and data producer responses. This means that this diagram and subsequent association of incoming message with the data producer and outgoing message with the data consumer should be changed to reflect the bi-directional nature of the OPES flow. > OPES processor and callout server exchange messages. The > exchange is bi-directional. There is no clear client or > server role. There is no clear request/response message > category either. > I do not agree with that. Per OPES architecture processing is initiating by rules that are triggered by some event in the OPES flow (arrival of request or response). As a result of some rule evaluation OPES processor can make a request to OPES application (either in the same box or a callout server). This creates a clear client-server roles: OPES processor makes a request and callout server produces response. OPES callout protocol should reflect these roles. Each transaction is initiated by the client (OPES processor) and executed by the server (callout server). There are certain message types that are produced by the client and other (different) types produced by the server. Protocol state transitions also should clearly reflect these roles. For example each transaction starts from message from the transaction client and is terminated by from the transaction server. BTW, my initial confusion was due to the absence of transaction roles. In bi-directional exchange without defined roles each side has to specify transaction boundaries, but with predefined client-server roles it is enough to have one open and one close message, as in your example. Note that protocol requires definite roles only per transaction. Roles are set by the opening message and may be different for different transactions. For the OPES architecture when processing is initiated by incoming OPES processor messages (from producer or from consumer) OPES processor is always a client. These roles may be different only for metadata requests (e.g. callout server requests about consumer or provider preferences). But again each transaction has client and server. The question is: do we allow such requests in-band? - Oskar From owner-ietf-openproxy@mail.imc.org Sun Feb 23 05:30:27 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA05672 for ; Sun, 23 Feb 2003 05:30:26 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1NA8HS00657 for ietf-openproxy-bks; Sun, 23 Feb 2003 02:08:17 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1NA8Fd00652 for ; Sun, 23 Feb 2003 02:08:16 -0800 (PST) Received: from f09v-10-62.d1.club-internet.fr ([212.194.189.62] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18mt3O-0003in-00 for ietf-openproxy@imc.org; Sun, 23 Feb 2003 02:08:15 -0800 Message-Id: <5.2.0.9.0.20030223101504.042f7270@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Sun, 23 Feb 2003 10:15:29 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: RE: OPES protocol, pre-draft Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On 19:59 21/02/03, Alex Rousskov said: >On Fri, 21 Feb 2003, jfcm wrote: >blame your mail agent instead, for not auto-filling both TO and >CC headers :-). Sending at least one mail too much :-) > > 2. there is a smarter, pipe oriented, low cpu load approach with > > user interaction which corresponds to what I look for and which > > is actually like a network operating system. > >Perhaps it is just me, but I am still not quite sure what ONES is or >should be. Could you please give two very specific, real-world ONES >examples? Clearly, if there is a smarter, lower-load architecture we >should at least consider it! You ask two things. - technical considerations: discussion about SOAP/noSOAP, redirecting or not. ICAP.2 etc. One vision is continuity with existing solutions, the other is a new vision related to a global optimisation.. - applications examples: - SMTP/FTP etc. support not HTTP to start with - redirects with request of agreement by the user - redirects/duplication upon thrid party decision (ex. write tapping) and all the interactions ONES/user/hosts/third parties. whicha re not from what I understand part of OPES. jfc From owner-ietf-openproxy@mail.imc.org Sun Feb 23 05:35:16 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA05726 for ; Sun, 23 Feb 2003 05:35:13 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1NA8KX00666 for ietf-openproxy-bks; Sun, 23 Feb 2003 02:08:20 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1NA8Jd00661 for ; Sun, 23 Feb 2003 02:08:19 -0800 (PST) Received: from f09v-10-62.d1.club-internet.fr ([212.194.189.62] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18mt3Q-0003in-00 for ietf-openproxy@imc.org; Sun, 23 Feb 2003 02:08:16 -0800 Message-Id: <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Sun, 23 Feb 2003 11:14:20 +0100 To: "OPES WG" From: jfcm Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: References: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="=====================_8421784==.ALT" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=====================_8421784==.ALT Content-Type: text/plain; charset="us-ascii"; format=flowed I am sorry, but I find this terminology confusing and limiting the analysis. We are talking about services. Theses services are called by dispatchers according to rules. If there is a service there is a user the service is delivered to and a host the services is delivered on behalf of. My understanding from IAB and first draft is that OPES correspond to the following sheme under HTTP: Service User ----> dispatcher -----> Service Organizer ^ | callout protocol v service provider As Oskar says, the traffic can be bidrectional. So I do not really know who the data consumer and the data producer are: Service User <----> dispatcher <----> Service Organizer ^ | callout protocol v service provider Others want to support SMTP etc. Others want the system to be able to fork. Service User <----> dispatcher <----> Service Organizer / ^ \ Other user / | \ Other Host v service provider My own reading is that this dialog system conducted by the dispatcher should become a polylogal system, hence "open" in my "open extended network system" naming. This is also why I call them "network" services while IAB specifies clearly "edge". And why I call them "extended" (in comparison with "basic" and "enhanced", because they relate users with as well as what you call data producers and with other data users) Because - as Oskar points it in saying it is bidireactional - I know who are: - the user and the producer in an access connection - the caller and the calleed party in a communication - but I do not know who are the user/producer nor the caller and callee in a relation - all the more in a distributed networked relations among a community/market. I can at most differentiate different classes of users (of of their agents) and groups of virtual services (organizers). In this what counts is not what is installed or proposed, but what I am allowed to use at a given time. And the very organizing host (supervisor) may be itself subject to in the flow changes (example: the proposed service promotes me on-the-fly to an "admin" status). To follow the same presentation as above: Supervisor or Service User <-----> dispatcher <----> Service Organizer / ^ ^ \ Other user / | | \ Other users v v service <--> server <----> service controler /provider dispatcher provider external user / ^ /^ etc.. etc.. | / | v / v etc. / alarms,logs / etc / other ONES domains At 21:08 22/02/03, Oskar Batuner wrote: >Hi Alex, >Here are some comments: > > > 0. Introduction > > > > draft-ietf-opes-protocol-reqs-03.txt defines the following > > information flow: > > > > data provider --(original application message)--> > > -- [ OPES magic ] --> > > --(produced application messages)--> data consumers > > > >I think both protocol requirements and architecture support OPES >processing in bi-directional message flow, both for data consumer >requests and data producer responses. This means that this diagram >and subsequent association of incoming message with the data producer >and outgoing message with the data consumer should be changed to >reflect the bi-directional nature of the OPES flow. True. But this only permits a dialog: ask the user to accept a redirection. It should support polylog (example: conditional access to be accepted. Ex. the redirection costs more and depending on the caller, an external authority (bank) is to permit it which may want to call the caller for extra warranties. > > OPES processor and callout server exchange messages. The > > exchange is bi-directional. There is no clear client or > > server role. There is no clear request/response message > > category either. > > > >I do not agree with that. Per OPES architecture processing is initiating >by rules that are triggered by some event in the OPES flow (arrival of >request or response). As a result of some rule evaluation OPES processor >can make a request to OPES application (either in the same box or a callout >server). This creates a clear client-server roles: OPES processor makes a >request and callout server produces response. There is a link between the dispatcher and the service. But none is the boss. There may be negotiations (both ends) or flow control (the service adding a rule in the dispatcher to block requests until it is ready again), etc.. A service may be linked to serveral dispatchers and a dispatcher to several identical/similar services. >OPES callout protocol should reflect these roles. Each transaction is >initiated by the client (OPES processor) and executed by the server >(callout server). There are certain message types that are produced by the >client and other (different) types produced by the server. Protocol state >transitions also should clearly reflect these roles. For example each >transaction starts from message from the transaction client >and is terminated by from the transaction server. BTW, my >initial confusion was due to the absence of transaction roles. In >bi-directional >exchange without defined roles each side has to specify transaction >boundaries, but >with predefined client-server roles it is enough to have one open and one >close message, as in your example. There is a client/server role within an accepted on-going cession/transaction. But if the link is broken what are the recovery decisions? The service may be to contact the user and rebuild the link through another dispatcher? >Note that protocol requires definite roles only per transaction. Roles are set >by the opening message and may be different for different transactions. True. >For the OPES architecture when processing is initiated by incoming OPES >processor >messages (from producer or from consumer) OPES processor is always a client. >These >roles may be different only for metadata requests (e.g. callout server >requests >about consumer or provider preferences). But again each transaction has client >and server. If I read you correctly you talk about two different client/server systems. One is producer/consumer, the other one is processor/server. And you say that depending on transactions the roles may be changed? >The question is: do we allow such requests in-band? May I suggest a question in response? How do we zap and abort the transactions? Should not the response be the same? jfc --=====================_8421784==.ALT Content-Type: text/html; charset="us-ascii" I am sorry, but I find this terminology confusing and limiting the analysis.

We are talking about services. Theses services are called by dispatchers according to rules. If there is a service there is a user the service is delivered to and a host the services is delivered on behalf of. My understanding from IAB and first draft is that OPES correspond to the following sheme under HTTP:

Service User ----> dispatcher -----> Service Organizer
                        ^
                        |  callout protocol
                        v
                     service
                     provider

As Oskar says, the traffic can be bidrectional. So I do not really know who the data consumer and the data producer are:

Service User <----> dispatcher <----> Service Organizer
                        ^
                        |  callout protocol
                        v
                     service
                     provider


Others want to support SMTP etc. Others want the system to be able to fork.

Service User <----> dispatcher <----> Service Organizer
                  /     ^       \
Other user      /       |         \    Other Host
                        v
                      service
                      provider


My own reading is that this dialog system conducted by the dispatcher should become a polylogal system, hence "open" in my "open extended network system" naming. This is also why I call them "network" services while IAB specifies clearly "edge". And why I call them "extended" (in comparison with "basic" and "enhanced", because they relate users with as well as what you call data producers and with other data users)

Because - as Oskar points it in saying it is bidireactional - I know who are:
- the user and the producer in an access connection
- the caller and the calleed party in a communication
- but I do not know who are the user/producer nor the caller and callee in a relation - all the more in a distributed networked relations among a community/market. I can at most differentiate different classes of users (of of their agents) and groups of virtual services (organizers). In this what counts is not what is installed or proposed, but what I am allowed to use at a given time. And the very organizing host (supervisor) may be itself subject to in the flow changes (example: the proposed service promotes me on-the-fly to an "admin" status).

To follow the same presentation as above:

                                       Supervisor or
Service User <-----> dispatcher <----> Service Organizer
                   /  ^      ^  \     
Other user       /    |      |    \  Other users
                      v      v
                service <--> server  <----> service
controler     /provider      dispatcher     provider
external user /                  ^            /^                                         
etc.. etc..                      |          /  |
                                 v        /    v
                                etc.    /   alarms,logs                                            
                                      /     etc
                                    /
                 other ONES domains


At 21:08 22/02/03, Oskar Batuner wrote:
Hi Alex,
Here are some comments:

> 0. Introduction
>
> draft-ietf-opes-protocol-reqs-03.txt defines the following
> information flow:
>
>   data provider --(original application message)-->
>   -- [ OPES magic ] -->
>   --(produced application messages)--> data consumers
>

I think both protocol requirements and architecture support OPES
processing in bi-directional message flow, both for data consumer
requests and data producer responses. This means that this diagram
and subsequent association of incoming message with the data producer
and outgoing message with the data consumer should be changed to
reflect the bi-directional nature of the OPES flow.

True. But this only permits a dialog: ask the user to accept
a redirection. It should support polylog (example: conditional
access to be accepted. Ex. the redirection costs more and
depending on the caller, an external authority (bank) is to
permit it which may want to call the caller for extra warranties.

> OPES processor and callout server exchange messages. The
> exchange is bi-directional. There is no clear client or
> server role.  There is no clear request/response message
> category either.
>

I do not agree with that. Per OPES architecture processing is initiating
by rules that are triggered by some event in the OPES flow (arrival of
request or response). As a result of some rule evaluation OPES processor
can make a request to OPES application (either in the same box or a callout
server). This creates a clear client-server roles: OPES processor makes a
request and callout server produces response.

There is a link between the dispatcher and the service. But none is
the boss. There may be negotiations (both ends) or flow control
(the service adding a rule in the dispatcher to block requests until
it is ready again), etc.. A service may be linked to serveral
dispatchers and a dispatcher to several identical/similar services.

OPES callout protocol should reflect these roles. Each transaction is
initiated by the client (OPES processor) and executed by the server
(callout server). There are certain message types that are produced by the
client and other (different) types produced by the server. Protocol state
transitions also should clearly reflect these roles. For example each
transaction starts from <xaction-start> message from the transaction client
and is terminated by <xaction-end> from the transaction server. BTW, my
initial confusion was due to the absence of transaction roles. In bi-directional
exchange without defined roles each side has to specify transaction boundaries, but
with predefined client-server roles it is enough to have one open and one
close message, as in your example.

There is a client/server role within an accepted on-going
cession/transaction. But if the link is broken what are the
recovery decisions? The service may be to contact the
user and rebuild the link through another dispatcher?

Note that protocol requires definite roles only per transaction. Roles are set
by the opening message and may be different for different transactions.

True.

For the OPES architecture when processing is initiated by incoming OPES
processor
messages (from producer or from consumer) OPES processor is always a client.
These
roles may be different only for metadata requests (e.g. callout server
requests
about consumer or provider preferences). But again each transaction has client
and server.

If I read you correctly you talk about two different client/server systems.
One is producer/consumer, the other one is processor/server.
And you say that depending on transactions the roles may be changed?

The question is: do we allow such requests in-band?

May I suggest a question in response? How do we zap
and abort the transactions? Should not the response be
the same?

jfc
--=====================_8421784==.ALT-- From owner-ietf-openproxy@mail.imc.org Mon Feb 24 10:54:09 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA17445 for ; Mon, 24 Feb 2003 10:54:07 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1OFRa109350 for ietf-openproxy-bks; Mon, 24 Feb 2003 07:27:36 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1OFRZd09344 for ; Mon, 24 Feb 2003 07:27:35 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1OFROG12231; Mon, 24 Feb 2003 10:27:25 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6XLW1>; Mon, 24 Feb 2003 10:27:25 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404F7A813@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: jfcm , ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft Date: Mon, 24 Feb 2003 10:27:22 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2DC19.39D5A0F0" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2DC19.39D5A0F0 Content-Type: text/plain hi, see inside abbie > -----Original Message----- > From: jfcm [mailto:info@utel.net] > Sent: Sunday, February 23, 2003 4:15 AM > To: ietf-openproxy@imc.org > Subject: RE: OPES protocol, pre-draft > > > > On 19:59 21/02/03, Alex Rousskov said: > >On Fri, 21 Feb 2003, jfcm wrote: > >blame your mail agent instead, for not > auto-filling both TO > >and SNIP > You ask two things. > > - technical considerations: discussion about SOAP/noSOAP, > redirecting or not. ICAP.2 etc. One vision is continuity with > existing solutions, the other is a new vision related to a > global optimisation.. > > - applications examples: > - SMTP/FTP etc. support not HTTP to start with > - redirects with request of agreement by the user > - redirects/duplication upon thrid party decision (ex. > write tapping) > and all the interactions ONES/user/hosts/third parties. > whicha re not from what I understand part of OPES. > jfc very good point. The example protocol in the architecture document is HTTP. So our first scenarios must be based on HTTP. By the way, it seems to me that the current thinking in the pre-draft-01 work is that the OPES processor is always fwding messages to the callout server. This may not be the case!! I would like us to consider this scenario. 1. Gold, silver and copper services provided by a service provider 2. Gold package, looks into user prefernces (stored either locally or thru a URI or whatever) 3. User trust the OPES processor with his financial/private data, willing to expose some of the preferences to selected companies. 4. User can log on with multiple devices (wireless, PC) 5. User requires language translation (or location based service), that is conditional on the access device 6. user is willing to pay for translation based on various criteria and he/she prefers company X. Now, company X is not willing to implement OCP (OPES Callout Protocol), but company X provide the translation as a Web Service. How would the predraft protocol would be used here. Is the OPES provider out of luck. Here the customer can be a Multimillion dollar company. Abbie ------_=_NextPart_001_01C2DC19.39D5A0F0 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES protocol, pre-draft

hi,

see inside
abbie


> -----Original Message-----
> From: jfcm [mailto:info@utel.net]
> Sent: Sunday, February 23, 2003 4:15 AM
> To: ietf-openproxy@imc.org
> Subject: RE: OPES protocol, pre-draft
>
>
>
> On 19:59 21/02/03, Alex Rousskov said:
> >On Fri, 21 Feb 2003, jfcm wrote:
> ><offtopic>blame your mail agent = instead, for not
> auto-filling both TO
> >and

SNIP

> You ask two things.
>
> - technical considerations: discussion about = SOAP/noSOAP,
>    redirecting or not. ICAP.2 = etc. One vision is continuity with
>    existing solutions, the other = is a new vision related to a
>    global optimisation..
>
> - applications examples:
>    - SMTP/FTP etc. support not = HTTP to start with
>    - redirects with request of = agreement by the user
>    - redirects/duplication upon = thrid party decision (ex.
> write tapping)
>     and all the = interactions ONES/user/hosts/third parties.
>     whicha re not from what = I understand part of OPES.
> jfc

very good point. The example protocol in the = architecture document is HTTP. So our first scenarios must be based on = HTTP.

By the way, it seems to me that the current thinking = in the pre-draft-01 work is that the OPES processor is always fwding = messages to the callout server. This may not be the case!!

I would like us to consider this scenario.

1. Gold, silver and copper services provided by a = service provider
2. Gold package, looks into user prefernces (stored = either locally or thru a URI or whatever)
3. User trust the OPES processor with his = financial/private data, willing to expose some of the preferences to = selected companies.

4. User can log on with multiple devices (wireless, = PC)
5. User requires language translation (or location = based service), that is conditional on the access device
6. user is willing to pay for translation based on = various criteria and he/she prefers company X.

Now, company X is not willing to implement OCP (OPES = Callout Protocol), but company X provide the translation as a Web = Service.

How would the predraft protocol would be used here. = Is the OPES provider out of luck. Here the customer can be a = Multimillion dollar company.


Abbie













------_=_NextPart_001_01C2DC19.39D5A0F0-- From owner-ietf-openproxy@mail.imc.org Mon Feb 24 12:38:03 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA19979 for ; Mon, 24 Feb 2003 12:38:02 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1OHBx114989 for ietf-openproxy-bks; Mon, 24 Feb 2003 09:11:59 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1OHBvd14985 for ; Mon, 24 Feb 2003 09:11:57 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1OHBxeM005902 for ; Mon, 24 Feb 2003 10:11:59 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1OHBxEq005901; Mon, 24 Feb 2003 10:11:59 -0700 (MST) (envelope-from rousskov) Date: Mon, 24 Feb 2003 10:11:59 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft In-Reply-To: <87609AFB433BD5118D5E0002A52CD75404F7A813@zcard0k6.ca.nortel.com> Message-ID: References: <87609AFB433BD5118D5E0002A52CD75404F7A813@zcard0k6.ca.nortel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Mon, 24 Feb 2003, Abbie Barbir wrote: > The example protocol in the architecture document is HTTP. So our > first scenarios must be based on HTTP. I do not think the order is important here. Overall application protocol coverage is important. Yes, we MUST support HTTP. However, we should try to support other popular protocols. There were several SMTP-based usage cases discussed on this list. There are probably other protocols worth keeping in mind (FTP, IM flavors?). It is much easier to support N protocols from the start then to support one and then retrofit support for N-1 later. > By the way, it seems to me that the current thinking in the > pre-draft-01 work is that the OPES processor is always fwding > messages to the callout server. This may not be the case!! Pre-draft-01 is about OPES callout protocol (for now). What gets forwarded to the callout server is about OPES rules. Thus, the current draft does not talk about forwarding decisions at all. The draft covers what happens _after_ the decision to forward has been made. Hilarie Orman may have asked about partial forwarding support. See predraft's TODO list. Partial forwarding is not yet supported because I need Hilarie's clarification on what exactly she is after. > I would like us to consider this scenario. > > 1. Gold, silver and copper services provided by a service provider > 2. Gold package, looks into user prefernces (stored either locally or thru a > URI or whatever) > 3. User trust the OPES processor with his financial/private data, willing to > expose some of the preferences to selected companies. > 4. User can log on with multiple devices (wireless, PC) > 5. User requires language translation (or location based service), that is > conditional on the access device > 6. user is willing to pay for translation based on various criteria and > he/she prefers company X. > > Now, company X is not willing to implement OCP (OPES Callout > Protocol), but company X provide the translation as a Web Service. > > How would the predraft protocol would be used here. Is the OPES > provider out of luck. Here the customer can be a Multimillion dollar > company. I must be missing something important here. You are saying that "X is not willing to implement OCP" and then asking "how the predraft [OCP] protocol would be used here". If X does not support OCP, predraft will not apply. Apparently, X is using some other protocol to implement language translation. If the customer is willing to pay for that other protocol (or just does not care), then X will serve the customer using that other protocol. If the customer is not willing (perhaps because the other protocol does not provide OPES-grade privacy enforcement), then X will not get customer's business. Moreover, content provider might want and be able to prohibit non-OPES-translations if client-side software is willing to enforce such restrictions by checking digital signatures and such. I am not answering your question though, am I? Thanks, Alex. From owner-ietf-openproxy@mail.imc.org Mon Feb 24 13:18:30 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA21440 for ; Mon, 24 Feb 2003 13:18:27 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1OHsCi17483 for ietf-openproxy-bks; Mon, 24 Feb 2003 09:54:12 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1OHsBd17479 for ; Mon, 24 Feb 2003 09:54:11 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1OHrxG26707; Mon, 24 Feb 2003 12:54:00 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6XQ1S>; Mon, 24 Feb 2003 12:54:00 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404F7AA30@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft Date: Mon, 24 Feb 2003 12:53:52 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2DC2D.B1189ED8" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2DC2D.B1189ED8 Content-Type: text/plain Alex, I would like us to start looking into sporting services (Web Services). The example, may require (for broader business/deployment) that an OPES processor may need to support SOAP. I really would like us to start looking into that. So far the Pre-draft discussion developes a protocol that could be easily implemented as a SOAP one. we just need few elements, such an an HTTP element (encpasulates HTTP partial or full), a Responce-Required element etc... Basically, one class of elements can incoporate all of the features of the pre-draft. However, we can add elements that allow for 1. Partial/full controlled access to a profile (user) 2. Partial/full access to policy 3. content-trace-route element (this one is optional) We should look into not requiring close coupling between the Callout Server service provider and the OPES provider in terms that they are bound to a spcific OCP protocol that can only be used to sell or provide a service. IF we look at SOAP, we can in thoery allow for any web service service's provider to become a Calllout Server. (with the proper OPES framework in mind, of course) Abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Monday, February 24, 2003 12:12 PM > To: ietf-openproxy@imc.org > Subject: RE: OPES protocol, pre-draft > > > > On Mon, 24 Feb 2003, Abbie Barbir wrote: > > > The example protocol in the architecture document is HTTP. So our > > first scenarios must be based on HTTP. > > I do not think the order is important here. Overall > application protocol coverage is important. Yes, we MUST > support HTTP. However, we should try to support other popular > protocols. There were several SMTP-based usage cases > discussed on this list. There are probably other protocols > worth keeping in mind (FTP, IM flavors?). It is much easier > to support N protocols from the start then to support one and > then retrofit support for N-1 later. > > > By the way, it seems to me that the current thinking in the > > pre-draft-01 work is that the OPES processor is always > fwding messages > > to the callout server. This may not be the case!! > > Pre-draft-01 is about OPES callout protocol (for now). What > gets forwarded to the callout server is about OPES rules. > Thus, the current draft does not talk about forwarding > decisions at all. The draft covers what happens _after_ the > decision to forward has been made. > > Hilarie Orman may have asked about partial forwarding > support. See predraft's TODO list. Partial forwarding is not > yet supported because I need Hilarie's clarification on what > exactly she is after. > > > I would like us to consider this scenario. > > > > 1. Gold, silver and copper services provided by a service > provider 2. > > Gold package, looks into user prefernces (stored either locally or > > thru a URI or whatever) 3. User trust the OPES processor with his > > financial/private data, willing to expose some of the > preferences to > > selected companies. 4. User can log on with multiple devices > > (wireless, PC) 5. User requires language translation (or location > > based service), that is conditional on the access device > > 6. user is willing to pay for translation based on various > criteria and > > he/she prefers company X. > > > > Now, company X is not willing to implement OCP (OPES Callout > > Protocol), but company X provide the translation as a Web Service. > > > > How would the predraft protocol would be used here. Is the OPES > > provider out of luck. Here the customer can be a > Multimillion dollar > > company. > > I must be missing something important here. You are saying > that "X is not willing to implement OCP" and then asking "how > the predraft [OCP] protocol would be used here". If X does > not support OCP, predraft will not apply. Apparently, X is > using some other protocol to implement language translation. > If the customer is willing to pay for that other protocol (or > just does not care), then X will serve the customer using > that other protocol. If the customer is not willing (perhaps > because the other protocol does not provide OPES-grade > privacy enforcement), then X will not get customer's > business. Moreover, content provider might want and be able > to prohibit non-OPES-translations if client-side software is > willing to enforce such restrictions by checking digital > signatures and such. > > I am not answering your question though, am I? > > Thanks, > > Alex. > ------_=_NextPart_001_01C2DC2D.B1189ED8 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES protocol, pre-draft

Alex,

I would like us to start looking into sporting = services (Web Services).
The example, may require (for broader = business/deployment) that an OPES processor may need to support = SOAP.

I really would like us to start looking into = that.
So far the Pre-draft discussion developes a protocol = that could be easily implemented as a SOAP one.

we just need few elements, such an an HTTP element = (encpasulates HTTP partial or full), a Responce-Required element etc... = Basically, one class of elements can incoporate all of the features of = the pre-draft.

However, we can add elements that allow for
1. Partial/full controlled access to a profile = (user)
2. Partial/full access to policy
3. content-trace-route element (this one is = optional)


We should look into not requiring close coupling = between the Callout Server service provider and the OPES provider in = terms that they are bound to a spcific OCP protocol that can only be = used to sell or provide a service.

IF we look at SOAP, we can in thoery allow for any = web service service's provider to become a Calllout Server. (with the = proper OPES framework in mind, of course)

Abbie


> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Monday, February 24, 2003 12:12 PM
> To: ietf-openproxy@imc.org
> Subject: RE: OPES protocol, pre-draft
>
>
>
> On Mon, 24 Feb 2003, Abbie Barbir wrote:
>
> > The example protocol in the architecture = document is HTTP. So our
> > first scenarios must be based on = HTTP.
>
> I do not think the order is important here. = Overall
> application protocol coverage is = important.  Yes, we MUST
> support HTTP. However, we should try to support = other popular
> protocols. There were several SMTP-based usage = cases
> discussed on this list. There are probably = other protocols
> worth keeping in mind (FTP, IM flavors?). It is = much easier
> to support N protocols from the start then to = support one and
> then retrofit support for N-1 later.
>
> > By the way, it seems to me that the = current thinking in the
> > pre-draft-01 work is that the OPES = processor is always
> fwding messages
> > to the callout server. This may not be the = case!!
>
> Pre-draft-01 is about OPES callout protocol = (for now). What
> gets forwarded to the callout server is about = OPES rules.
> Thus, the current draft does not talk about = forwarding
> decisions at all. The draft covers what happens = _after_ the
> decision to forward has been made.
>
> Hilarie Orman may have asked about partial = forwarding
> support. See predraft's TODO list. Partial = forwarding is not
> yet supported because I need Hilarie's = clarification on what
> exactly she is after.
>
> > I would like us to consider this = scenario.
> >
> > 1. Gold, silver and copper services = provided by a service
> provider 2.
> > Gold package, looks into user prefernces = (stored either locally or
> > thru a URI or whatever) 3. User trust the = OPES processor with his
> > financial/private data, willing to expose = some of the
> preferences to
> > selected companies. 4. User can log on = with multiple devices
> > (wireless, PC) 5. User requires language = translation (or location
> > based service), that is conditional on the = access device
> > 6. user is willing to pay for translation = based on various
> criteria and
> > he/she prefers company X.
> >
> > Now, company X is not willing to implement = OCP (OPES Callout
> > Protocol), but company X provide the = translation as a Web Service.
> >
> > How would the predraft protocol would be = used here. Is the OPES
> > provider out of luck. Here the customer = can be a
> Multimillion dollar
> > company.
>
> I must be missing something important here. You = are saying
> that "X is not willing to implement = OCP" and then asking "how
> the predraft [OCP] protocol would be used = here". If X does
> not support OCP, predraft will not apply. = Apparently, X is
> using some other protocol to implement language = translation.
> If the customer is willing to pay for that = other protocol (or
> just does not care), then X will serve the = customer using
> that other protocol. If the customer is not = willing (perhaps
> because the other protocol does not provide = OPES-grade
> privacy enforcement), then X will not get = customer's
> business.  Moreover, content provider = might want and be able
> to prohibit non-OPES-translations if = client-side software is
> willing to enforce such restrictions by = checking digital
> signatures and such.
>
> I am not answering your question though, am = I?
>
> Thanks,
>
> Alex.
>

------_=_NextPart_001_01C2DC2D.B1189ED8-- From owner-ietf-openproxy@mail.imc.org Mon Feb 24 13:48:49 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22364 for ; Mon, 24 Feb 2003 13:48:46 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1OIMoU19157 for ietf-openproxy-bks; Mon, 24 Feb 2003 10:22:50 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1OIMnd19153 for ; Mon, 24 Feb 2003 10:22:49 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1OIMpeM007526 for ; Mon, 24 Feb 2003 11:22:51 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1OIMpBj007525; Mon, 24 Feb 2003 11:22:51 -0700 (MST) (envelope-from rousskov) Date: Mon, 24 Feb 2003 11:22:51 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Sat, 22 Feb 2003, Oskar Batuner wrote: > Here are some comments: > > > 0. Introduction > > > > draft-ietf-opes-protocol-reqs-03.txt defines the following > > information flow: > > > > data provider --(original application message)--> > > -- [ OPES magic ] --> > > --(produced application messages)--> data consumers > > > > I think both protocol requirements and architecture support OPES > processing in bi-directional message flow, both for data consumer > requests and data producer responses. This means that this diagram > and subsequent association of incoming message with the data > producer and outgoing message with the data consumer should be > changed to reflect the bi-directional nature of the OPES flow. Oskar, You are right. I apparently misinterpreted the "consumer/producer" terminology in the architecture draft. Fortunately, the protocol itself requires no serious adjustments because there are still application transactions, and there are still application messages that belong to those transactions. A couple of OPES messages names and definitions should be changed to bring them in sync with the architecture terminology, but these are cosmetic changes. In other words, the proposed protocol does not depend on how many application messages there are per application transaction or on their "direction". Draft-01 implies there are usually at least two application messages per application transaction (original and produced). Draft-02 will remove that implication and will use appropriate producer/consumer terminology from the architecture draft.. > > OPES processor and callout server exchange messages. The > > exchange is bi-directional. There is no clear client or > > server role. There is no clear request/response message > > category either. > > > > I do not agree with that. Per OPES architecture processing is > initiating by rules that are triggered by some event in the OPES > flow (arrival of request or response). As a result of some rule > evaluation OPES processor can make a request to OPES application > (either in the same box or a callout server). I agree with that :-). > This creates a clear client-server roles: OPES processor makes a > request and callout server produces response. While I think this is not the only possible terminology (it may not be enough to originate a transaction to be called a client), I do not want to argue about this and will try to change the Introduction wording according to your preferences. This terminology disagreement does not change the protocol. My original reasoning was to avoid "client" and "server" terms because OPES processor and callout server exchange many messages in the context of a single application transaction and those messages are not pure request/response sequences. But we can still call OPES processor a client; not a big deal. > OPES callout protocol should reflect these roles. Each transaction > is initiated by the client (OPES processor) and executed by the > server (callout server). There are certain message types that are > produced by the client and other (different) types produced by the > server. Protocol state transitions also should clearly reflect these > roles. I agree with that. Keep-alive and maintenance messages may be originated at the server, but this is minor. OPES transactions are initiated by OPES processor and executed by the callout server. > For example each transaction starts from message > from the transaction client Yes. > and is terminated by from the transaction server. This restriction is unnecessary and cannot be derived from the client-server model. It is perfectly normal for a client-server protocol to allow either side to terminate the transaction. Usually, the transaction will be terminated once one party signals the "end" and the other side receives that signal. In some protocols (e.g., SSL/TLS) the end of a transaction is a much more complex event. In OPES context, the client (OPES processor), may terminate the transaction by sending a message to the callout server. The callout server might send that message first (probably only under unusual conditions like overload). The two sides might even send concurrently! Also, please note that OPES protocol has to deal with three kinds of "end" messages: - end-of-data from the data source (does not imply that the work on the application message is done) - end-of-work for the application message (implies end-of-data but does not imply that the work on the application transaction is done) - end-of-work for the application transaction (implies end-of-data and end-of-message) > For the OPES architecture when processing is initiated by incoming > OPES processor messages (from producer or from consumer) OPES > processor is always a client. These roles may be different only for > metadata requests (e.g. callout server requests about consumer or > provider preferences). But again each transaction has client and > server. I understand your desire to call OPES processor a client and callout server a server in the context of an OPES transaction. I will adjust predraft wording accordingly. > The question is: do we allow such requests in-band? If I understand the question correctly, this is purely an OPES connection management issue. We can, for example, mark some connections "for management messages only" by sending a special message on that connection once the connection is open by the OPES processor. Does anybody have a preference/recommendation? Should transaction-independent messages be isolated to their own OPES transport connections? If yes, should this be a MAY for OPES processor to decide or a MUST that all processors will follow? My suggestion is that it make it a MAY. I do not see a reason to mandate presence of control-only (out-of-band) connections, but I can see a reason to have such connections for priority message handling. Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Mon Feb 24 14:01:23 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA22730 for ; Mon, 24 Feb 2003 14:01:22 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1OIb9719900 for ietf-openproxy-bks; Mon, 24 Feb 2003 10:37:09 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1OIb8d19896 for ; Mon, 24 Feb 2003 10:37:08 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1OIbAeM007867 for ; Mon, 24 Feb 2003 11:37:10 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1OIbA4k007866; Mon, 24 Feb 2003 11:37:10 -0700 (MST) (envelope-from rousskov) Date: Mon, 24 Feb 2003 11:37:10 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Sun, 23 Feb 2003, jfcm wrote: > Supervisor or > Service User <-----> dispatcher <----> Service Organizer > / ^ ^ \ > Other user / | | \ Other users > v v > service <--> server <----> service > controler /provider dispatcher provider > external user > / ^ /^ > etc.. etc.. | / | > v / v > etc. / alarms,logs > > / etc > / > other ONES domains > My understanding is that the above is more-or-less allowed by OPES architecture (as long as it can support OPES requirements). IIRC, callout servers can delegate their functions and, hence, essentially become "server dispatcher" on your diagram. For a second, let's try to ignore what the terminology may or may not imply. Can you point out specific desirable things that current OPES architecture, OCP requirements, and predraft protocol do NOT allow you to do? If you can name specific areas that cannot be supported by OPES, we can discuss whether those areas should be supported (or perhaps already are supported!). Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Mon Feb 24 15:37:25 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA25766 for ; Mon, 24 Feb 2003 15:37:24 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1OKDip26242 for ietf-openproxy-bks; Mon, 24 Feb 2003 12:13:44 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1OKDhd26238 for ; Mon, 24 Feb 2003 12:13:43 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1OKDjeM010986 for ; Mon, 24 Feb 2003 13:13:45 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1OKDj0Y010985; Mon, 24 Feb 2003 13:13:45 -0700 (MST) (envelope-from rousskov) Date: Mon, 24 Feb 2003 13:13:45 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: OPES transport protocol In-Reply-To: <87609AFB433BD5118D5E0002A52CD75404F7AA30@zcard0k6.ca.nortel.com> Message-ID: References: <87609AFB433BD5118D5E0002A52CD75404F7AA30@zcard0k6.ca.nortel.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Mon, 24 Feb 2003, Abbie Barbir wrote: > I would like us to start looking into sporting services (Web > Services). The example, may require (for broader > business/deployment) that an OPES processor may need to support > SOAP. > > I really would like us to start looking into that. So far the > Pre-draft discussion developes a protocol that could be easily > implemented as a SOAP one. > > we just need few elements, such an an HTTP element (encpasulates > HTTP partial or full), a Responce-Required element etc... Basically, > one class of elements can incoporate all of the features of the > pre-draft. > > However, we can add elements that allow for > 1. Partial/full controlled access to a profile (user) > 2. Partial/full access to policy > 3. content-trace-route element (this one is optional) It is great that the predraft protocol can be implemented on top of SOAP. That means our options regarding OPES transport are still wide open; all transport protocols mentioned on this list can be supported: BEEP, HTTP, and SOAP. At some point, we need to decide whether to standardize a single transport protocol binding (e.g., SOAP) OR allow multiple bindings (e.g., HTTP or SOAP) with one "OPES binding" spec (RFC?) per supported transport protocol. The one-binding approach is simpler but the N-binding approach is more flexible. > We should look into not requiring close coupling between the Callout > Server service provider and the OPES provider in terms that they are > bound to a spcific OCP protocol that can only be used to sell or > provide a service. > > IF we look at SOAP, we can in thoery allow for any web service > service's provider to become a Calllout Server. (with the proper > OPES framework in mind, of course) IETF in general and OPES WG in particular cannot dictate what a service provider can sell. Clearly, there will be competing protocols and approaches on the market, and OPES would have to fight its way to the world dominance. We have to work in OPES context: a service provider either supports OPES or it does not. We cannot accommodate service providers that do not use OPES (e.g., those that use raw SOAP with non-OPES extensions). We can only try to make OPES attractive enough so that more service providers support it. It would be great if current service providers can switch to (or add support for) OPES without much effort. If you can convincingly demonstrate that having SOAP binding will make transition to OPES simple, it would significantly increase chances that SOAP becomes an OPES binding. To make this case, we have to finish implementing the core of the protocol, I guess. Before that happens, it is difficult to discuss transitions from existing approaches since there is nothing to transition to! Finally, based on e-mails on this list, I expect some resistance to SOAP because SOAP is not an IETF standard. I do not know what the official IETF (IAB?) position is regarding developing IETF standards that rely on IETF-level but non-IETF standards. Are there historical precedents (e.g., standard-track RFCs) of IETF specs based on SOAP? If there is a political problem with SOAP, supporting multiple optional bindings (including SOAP) may help to avoid a deadly clash with IETF policies. Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Mon Feb 24 15:52:38 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA26232 for ; Mon, 24 Feb 2003 15:52:37 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1OKTtO26970 for ietf-openproxy-bks; Mon, 24 Feb 2003 12:29:55 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1OKTrd26966 for ; Mon, 24 Feb 2003 12:29:53 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1OKTueM011362 for ; Mon, 24 Feb 2003 13:29:56 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1OKTugL011361; Mon, 24 Feb 2003 13:29:56 -0700 (MST) (envelope-from rousskov) Date: Mon, 24 Feb 2003 13:29:56 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Sun, 23 Feb 2003, jfcm wrote: > How do we zap and abort the transactions? I assume you are talking about the situation where user request must stop at the OPES processor or the callout server due to rules/policies/etc. The predraft document describes how a callout server can abort an application transaction. The technique is based on changing the source and destination addresses: the source address becomes the address of the callout server. The destination address becomes the address of the application client (it has to even point to the same application connection if application protocol requires that). When OPES processor receives a OPES message with such addresses, it would start responding to the original client request instead of forwarding the request to the server. As you know, Hilarie Orman (and probably others), objected to giving callout servers ability to change source and destination addresses. This objection would need to be resolved by either convincing opponents that there are no good reasons to prohibit such a change OR adding new OCP mechanisms to allow callout servers to tell OPES processor: "you expected me to modify the request to be forwarded, but I am telling you to abort forwarding and respond with the following message instead" Note that in the current proposal, the semantics is somewhat different (abort is not an exceptional condition in the predraft): "you expected me to modify the application message; here is a modified version; forward it [back to the client]" Does the above answer your question? Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Mon Feb 24 17:18:22 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA28494 for ; Mon, 24 Feb 2003 17:18:21 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1OLrUM01842 for ietf-openproxy-bks; Mon, 24 Feb 2003 13:53:30 -0800 (PST) Received: from sccrmhc01.attbi.com (sccrmhc01.attbi.com [204.127.202.61]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1OLrQd01838 for ; Mon, 24 Feb 2003 13:53:26 -0800 (PST) Received: from oskar3 (h00a0c5e5337d.ne.client2.attbi.com[24.218.184.133]) by sccrmhc01.attbi.com (sccrmhc01) with SMTP id <200302242153230010031eoqe>; Mon, 24 Feb 2003 21:53:23 +0000 From: "Oskar Batuner" To: "OPES WG" Subject: RE: OPES protocol, pre-draft 01 Date: Mon, 24 Feb 2003 16:53:57 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit > -----Original Message----- > From: owner-ietf-openproxy@mail.imc.org > [mailto:owner-ietf-openproxy@mail.imc.org]On Behalf Of Alex Rousskov > Sent: Monday, February 24, 2003 1:23 PM > To: OPES WG > Subject: RE: OPES protocol, pre-draft 01 > > > > On Sat, 22 Feb 2003, Oskar Batuner wrote: [skip] > > For example each transaction starts from message > > from the transaction client > > Yes. > > > and is terminated by from the transaction server. > > This restriction is unnecessary and cannot be derived from the > client-server model. It is perfectly normal for a client-server > protocol to allow either side to terminate the transaction. Usually, > the transaction will be terminated once one party signals the "end" > and the other side receives that signal. In some protocols (e.g., > SSL/TLS) the end of a transaction is a much more complex event. > > In OPES context, the client (OPES processor), may terminate the > transaction by sending a message to the callout server. > The callout server might send that message first (probably only under > unusual conditions like overload). The two sides might even send > concurrently! > > Also, please note that OPES protocol has to deal with three kinds of > "end" messages: > - end-of-data from the data source > (does not imply that the work on the application message is > done) > - end-of-work for the application message > (implies end-of-data but does not imply that the work on the > application transaction is done) > - end-of-work for the application transaction > (implies end-of-data and end-of-message) > Here I'm looking at transaction boundaries that are defined by and . You are right that in general it is not necessary to limit the right to terminate transactions to the server. But in the OPES case > > ... processing is initiated by incoming > > OPES processor messages (from producer or from consumer) ... In such context OPES server has no apparent reason to terminate message processing - there are no possible events associated with this transaction coming from the OPES flow. After initial message is received and service is initiated only service application (in this case callout server) knows that transaction processing is finished. At least for the normal processing. There may be abnormal situations (e.g. OPES data consumer has closed the connection) when OPES processor becomes not interested in processing results and may wish to stop request processing to avoid resource wasting. Maybe we should have a special message for abnormal transaction termination? > > The question is: do we allow such requests in-band? > > If I understand the question correctly, this is purely an OPES > connection management issue. We can, for example, mark some > connections "for management messages only" by sending a special > message on that connection once the connection is open by the OPES > processor. Does anybody have a preference/recommendation? Should > transaction-independent messages be isolated to their own OPES > transport connections? If yes, should this be a MAY for OPES processor > to decide or a MUST that all processors will follow? In the architecture draft we are always separating OPES flow from the metadata exchange. We state that metadata transfers are out-of-scope for the OPES architecture. Of cause this statement does not preclude using the same callout protocol - or any other - for the metadata transfer. But if we permit mixing OPES data and metadata (like preferences, etc.) on the same channel - well, this looks at least inconsistent. > > My suggestion is that it make it a MAY. I do not see a reason to > mandate presence of control-only (out-of-band) connections, but I can > see a reason to have such connections for priority message handling. > Oskar From owner-ietf-openproxy@mail.imc.org Mon Feb 24 18:16:11 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA29813 for ; Mon, 24 Feb 2003 18:16:09 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1OMsGr03689 for ietf-openproxy-bks; Mon, 24 Feb 2003 14:54:16 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1OMsEd03684 for ; Mon, 24 Feb 2003 14:54:14 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1OMsHeM014687 for ; Mon, 24 Feb 2003 15:54:17 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1OMsHJa014686; Mon, 24 Feb 2003 15:54:17 -0700 (MST) (envelope-from rousskov) Date: Mon, 24 Feb 2003 15:54:17 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Mon, 24 Feb 2003, Oskar Batuner wrote: > Here I'm looking at transaction boundaries that are defined by > and . You are right that in general > it is not necessary to limit the right to terminate transactions to > the server. But in the OPES case > > > > ... processing is initiated by incoming > > > OPES processor messages (from producer or from consumer) ... > > In such context OPES server has no apparent reason to terminate > message processing - there are no possible events associated with > this transaction coming from the OPES flow. After initial message is > received and service is initiated only service application (in this > case callout server) knows that transaction processing is finished. > At least for the normal processing. There may be abnormal situations > (e.g. OPES data consumer has closed the connection) when OPES > processor becomes not interested in processing results and may wish > to stop request processing to avoid resource wasting. > > Maybe we should have a special message for abnormal transaction > termination? Abnormal termination is exactly what motivated me to allow messages to be sent by OPES processor. We could have a special message, but - the recipient (callout server) most likely does not really care why the transaction is suddenly ended; the reason field can be used for detailed logging/debugging if desired - the reason field(s) can already be used to indicate the kind of termination through application-specific status codes (normal, client errors, internal errors, overload, etc.) - there might be application protocols where "normal" termination can be detected by OPES processor; the notion of "normal termination" is, after all, application [protocol] specific. Overall, a more general message seems to cause no problems while having possibly desirable side-effects. Any objections? > In the architecture draft we are always separating OPES flow from > the metadata exchange. We state that metadata transfers are > out-of-scope for the OPES architecture. Of cause this statement does > not preclude using the same callout protocol - or any other - for > the metadata transfer. But if we permit mixing OPES data and > metadata (like preferences, etc.) on the same channel - well, this > looks at least inconsistent. I understand the intent -- keep OPES focused and small. Is there a precise definition of "metadata" or "OPES flow" though? For example, most will probably agree that service ID(s) should be attached to messages, but many will consider a service ID to be METAdata because the application protocol does not carry those. Another example: transferring a static set of preferences and rules should probably be out of OPES protocol scope. On the other hand, sending transaction-specific rules that affect, say, service application order may be in OPES scope. Same for specific user preferences and/or credentials? It seems to me that metadata specific to application transaction may be in scope and may be transmitted in-band. Alex. From owner-ietf-openproxy@mail.imc.org Mon Feb 24 19:26:10 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA01347 for ; Mon, 24 Feb 2003 19:26:09 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1ONqgA06052 for ietf-openproxy-bks; Mon, 24 Feb 2003 15:52:42 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1ONqfd06047 for ; Mon, 24 Feb 2003 15:52:41 -0800 (PST) Received: from f08v-10-110.d1.club-internet.fr ([212.194.165.110] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18nSOn-0002Nm-00 for ietf-openproxy@imc.org; Mon, 24 Feb 2003 15:52:41 -0800 Message-Id: <5.2.0.9.0.20030226004549.041e0d70@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Wed, 26 Feb 2003 00:55:58 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: References: <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On 21:29 24/02/03, Alex Rousskov said: >On Sun, 23 Feb 2003, jfcm wrote: > > > How do we zap and abort the transactions? > >I assume you are talking about the situation where user request must >stop at the OPES processor or the callout server due to >rules/policies/etc. No. The question was about parallel command pipe or not. My response was to try to see if we can have a consistent response. Obviously triggering a command should probably go the same way as aborting it. The idea of a piped protocol being accepted, you must zap the pipe if you want to abort. This means a priority somewhere in the processus. Either in the pipe with a zapper swallowing the data. Or a prioritized command pipe ordering to clear the four buffers. Then there is the message telling the server to abort and and ack to the requester. >The predraft document describes how a callout server can abort an >application transaction. The technique is based on changing the source >and destination addresses: the source address becomes the address of >the callout server. The destination address becomes the address of the >application client (it has to even point to the same application >connection if application protocol requires that). When OPES processor >receives a OPES message with such addresses, it would >start responding to the original client request instead of forwarding >the request to the server. I am not sure I grasp all the implications in all the cases. I need to make a .. model. And fully understand. The real problem is also that I do not always figure out what can be the osurce, the destination the processor, the server, the client, the producer :-) >As you know, Hilarie Orman (and probably others), objected to giving >callout servers ability to change source and destination addresses. >This objection would need to be resolved by either convincing >opponents that there are no good reasons to prohibit such a change OR >adding new OCP mechanisms to allow callout servers to tell OPES >processor: > > "you expected me to modify the request to be forwarded, but I > am telling you to abort forwarding and respond with the > following message instead" This is to abord forwarding. I was talking about command pipe, hence abordting the service, not necessarilythe forwarding. This is another option to add. >Note that in the current proposal, the semantics is somewhat different >(abort is not an exceptional condition in the predraft): > > "you expected me to modify the application message; here is > a modified version; forward it [back to the client]" idem? However simpler. The real issue is that at minimum there are four flows involved user/dispatcher/service/dispatcher/organizer in my own words instead of one. Following all the possiblities calls for an Excell spreadshit. jfc From owner-ietf-openproxy@mail.imc.org Tue Feb 25 00:00:43 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA06277 for ; Tue, 25 Feb 2003 00:00:41 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1P4Uvq13606 for ietf-openproxy-bks; Mon, 24 Feb 2003 20:30:57 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1P4Utd13602 for ; Mon, 24 Feb 2003 20:30:55 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1P4UxeM022361 for ; Mon, 24 Feb 2003 21:30:59 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1P4UxYL022360; Mon, 24 Feb 2003 21:30:59 -0700 (MST) (envelope-from rousskov) Date: Mon, 24 Feb 2003 21:30:59 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: <5.2.0.9.0.20030226004549.041e0d70@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> <5.2.0.9.0.20030226004549.041e0d70@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Wed, 26 Feb 2003, jfcm wrote: > No. The question was about parallel command pipe or not. My response > was to try to see if we can have a consistent response. Obviously > triggering a command should probably go the same way as aborting it. If I understand the above, that is what the current predraft proposal does. Triggering transaction processing on the other end uses the same mechanism as aborting that processing (an OPES message with the corresponding transaction ID xid). > The idea of a piped protocol being accepted, you must zap the pipe > if you want to abort. This means a priority somewhere in the > processus. Either in the pipe with a zapper swallowing the data. Or > a prioritized command pipe ordering to clear the four buffers. Priority processing is not required to terminate a transaction. Priority processing is an optimization -- it allows to terminate transaction on the other side _faster_. How much faster? Depends on how exactly one organizes priority queues and on how long those queues are at the abortion time. If you recall, the predraft protocol allows for out-of-order (i.e., prioritized) commands but does not document how the transport connections are handled (yet). If a OPES processor opens more connection to the callout server than there are concurrent application messages, then a message can be sent using "spare" [control] connections, possibly triggering a faster abort. I do not know what you mean by "zapper swallowing the data" in the pipe. Can you clarify, please? Since we are talking about TCP-like connections here, it may not be possible to "swallow" anything that was sent. The only reliable option is to ignore messages with unknown (e.g., already aborted) xid and/or terminate the corresponding transport connection. Perhaps that's what you meant by swallowing? > Then there is the message telling the server to abort and and ack to > the requester. An ACK would be a waste of resources unless you want to support OPES transaction resumption after the event of transport-layer connectivity loss. By "resumption", I mean resuming at the place of last synchronization, transparently to the consumer and producer. I doubt the benefits of transaction resumption outweigh increased protocol complexity. It seems to me that for application protocols we know about (HTTP and SMTP), an application transaction abort or an automated retry would be more appropriate solutions. Do you think OPES transactions should be resumable if the transport layer fails? > The real issue is that at minimum there are four flows involved > user/dispatcher/service/dispatcher/organizer in my own words instead > of one. Following all the possiblities calls for an Excell > spreadshit. In such cases, a higher level simplification or abstraction often works better than enumerating endless possibilities in a table (IMO). The proposed OCP protocol is such an abstraction (at this point in time) because it does not really care what the consumer, producer, source, or destinations are. The number of flows is not important as long as all flows are mostly independent and there is a well-known "forking" or "initial scheduling" point (the OPES processor in our case). The proposed protocol handles at least two flows per application transaction and should scale well with the number of flows. Alex. From owner-ietf-openproxy@mail.imc.org Tue Feb 25 10:02:15 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA02571 for ; Tue, 25 Feb 2003 10:02:13 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1PEW7x05804 for ietf-openproxy-bks; Tue, 25 Feb 2003 06:32:07 -0800 (PST) Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1PEW7d05799 for ; Tue, 25 Feb 2003 06:32:07 -0800 (PST) Received: from oskar3 (h00a0c5e5337d.ne.client2.attbi.com[24.218.184.133]) by rwcrmhc53.attbi.com (rwcrmhc53) with SMTP id <2003022514315205300g36i0e>; Tue, 25 Feb 2003 14:31:52 +0000 From: "Oskar Batuner" To: "Alex Rousskov" , "OPES WG" Subject: RE: OPES protocol, pre-draft 01 Date: Tue, 25 Feb 2003 09:32:27 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit > -----Original Message----- > From: owner-ietf-openproxy@mail.imc.org > [mailto:owner-ietf-openproxy@mail.imc.org]On Behalf Of Alex Rousskov > Sent: Monday, February 24, 2003 5:54 PM > To: OPES WG > Subject: RE: OPES protocol, pre-draft 01 > > > > On Mon, 24 Feb 2003, Oskar Batuner wrote: > > > Here I'm looking at transaction boundaries that are defined by > > and . You are right that in general > > it is not necessary to limit the right to terminate transactions to > > the server. But in the OPES case > > > > > > ... processing is initiated by incoming > > > > OPES processor messages (from producer or from consumer) ... > > > > In such context OPES server has no apparent reason to terminate > > message processing - there are no possible events associated with > > this transaction coming from the OPES flow. After initial message is > > received and service is initiated only service application (in this > > case callout server) knows that transaction processing is finished. > > At least for the normal processing. There may be abnormal situations > > (e.g. OPES data consumer has closed the connection) when OPES > > processor becomes not interested in processing results and may wish > > to stop request processing to avoid resource wasting. > > > > Maybe we should have a special message for abnormal transaction > > termination? > > Abnormal termination is exactly what motivated me to allow > messages to be sent by OPES processor. We could have a > special message, but > > - the recipient (callout server) most likely does not > really care why the transaction is suddenly ended; the > reason field can be used for detailed logging/debugging > if desired > > - the reason field(s) can already be used to indicate > the kind of termination through application-specific > status codes (normal, client errors, internal errors, > overload, etc.) > > - there might be application protocols where "normal" > termination can be detected by OPES processor; > the notion of "normal termination" is, after all, > application [protocol] specific. > > Overall, a more general message seems to cause no > problems while having possibly desirable side-effects. Any objections? Abnormal termination is different from normal termination (and all other normal messages) in the sense that it switches context from message processing to termination and cleanup. I think this difference justifies presence of a separate message. In some sense it is a question of preferences - as you have correctly pointed out difference between normal and abnormal termination messages can be determined by some additional reason code in the same message. I prefer to keep an explicit difference. Another difference is that abnormal termination message may arrive at any moment, other messages are limited by the processing context (protocol state machine). > > > In the architecture draft we are always separating OPES flow from > > the metadata exchange. We state that metadata transfers are > > out-of-scope for the OPES architecture. Of cause this statement does > > not preclude using the same callout protocol - or any other - for > > the metadata transfer. But if we permit mixing OPES data and > > metadata (like preferences, etc.) on the same channel - well, this > > looks at least inconsistent. > > I understand the intent -- keep OPES focused and small. Is there a > precise definition of "metadata" or "OPES flow" though? For example, > most will probably agree that service ID(s) should be attached to > messages, but many will consider a service ID to be > METAdata because the application protocol does not carry those. > > Another example: transferring a static set of preferences and rules > should probably be out of OPES protocol scope. On the other hand, > sending transaction-specific rules that affect, say, service > application order may be in OPES scope. Same for specific user > preferences and/or credentials? It seems to me that metadata specific > to application transaction may be in scope and may be transmitted > in-band. Yes, these is why I'm asking this question. We may go both ways. In many cases it may be very natural to transfer in-band metadata related to the current processing context. But as soon as you allow in-band metadata transfer you may not set a clear border - what is specific to the transaction and what is not. If we are allowed to send user preferences when message to/from that user is processed - why not send to remember these preferences on callout processor? And if callout processor keeps user's data - why not to send it them just once, in advance? Real decision is either yes (allow metadata transfer in-band) or no. Metadata transfer in a limited context is not a consistent architectural decision: all related protection measures mandated by the architecture MUST be implemented, but if they are - why to limit the context? Oskar From owner-ietf-openproxy@mail.imc.org Tue Feb 25 11:13:16 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA04901 for ; Tue, 25 Feb 2003 11:13:15 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1PFjGK12216 for ietf-openproxy-bks; Tue, 25 Feb 2003 07:45:16 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1PFjFd12212 for ; Tue, 25 Feb 2003 07:45:15 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1PFj7R17200; Tue, 25 Feb 2003 10:45:07 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6X0LV>; Tue, 25 Feb 2003 10:45:07 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404F7B1B5@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , ietf-openproxy@imc.org Subject: RE: OPES transport protocol Date: Tue, 25 Feb 2003 10:45:05 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2DCE4.DE12A6F6" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2DCE4.DE12A6F6 Content-Type: text/plain > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Monday, February 24, 2003 3:14 PM > To: ietf-openproxy@imc.org > Subject: OPES transport protocol > > > SNIP > > Finally, based on e-mails on this list, I expect some > resistance to SOAP because SOAP is not an IETF standard. I do > not know what the official IETF (IAB?) position is regarding > developing IETF standards that rely on IETF-level but > non-IETF standards. Are there historical precedents (e.g., > standard-track RFCs) of IETF specs based on SOAP? If there is > a political problem with SOAP, supporting multiple optional > bindings (including SOAP) may help to avoid a deadly clash > with IETF policies. > > > Thank you, > > Alex. > Yes, I would expect resistance to SOAP, but I may also expect resistance to any other protocols too. The first step that need to be answered is why not this protocl, why a new one etc.. There was some debate about SOAP before and the views varied. U can do a check on it by searching for SOAP on the ID-draft link on the IETF homepage. abbie ------_=_NextPart_001_01C2DCE4.DE12A6F6 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES transport protocol

> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Monday, February 24, 2003 3:14 PM
> To: ietf-openproxy@imc.org
> Subject: OPES transport protocol
>
>
>
SNIP
>
> Finally, based on e-mails on this list, I = expect some
> resistance to SOAP because SOAP is not an IETF = standard. I do
> not know what the official IETF (IAB?) position = is regarding
> developing IETF standards that rely on = IETF-level but
> non-IETF standards. Are there historical = precedents (e.g.,
> standard-track RFCs) of IETF specs based on = SOAP? If there is
> a political problem with SOAP, supporting = multiple optional
> bindings (including SOAP) may help to avoid a = deadly clash
> with IETF policies.
>
>
> Thank you,
>
> Alex.
>

Yes, I would expect resistance to SOAP, but I may = also expect resistance to any other protocols too.
The first step that need to be answered is why not = this protocl, why a new one etc..

There was some debate about SOAP before and the views = varied. U can do a check on it by searching for SOAP on the ID-draft = link on the IETF homepage.

abbie











------_=_NextPart_001_01C2DCE4.DE12A6F6-- From owner-ietf-openproxy@mail.imc.org Tue Feb 25 12:00:14 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA06458 for ; Tue, 25 Feb 2003 12:00:13 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1PGOsC14265 for ietf-openproxy-bks; Tue, 25 Feb 2003 08:24:54 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1PGOqd14260 for ; Tue, 25 Feb 2003 08:24:52 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1PGOseM039003 for ; Tue, 25 Feb 2003 09:24:54 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1PGOsxL039002; Tue, 25 Feb 2003 09:24:54 -0700 (MST) (envelope-from rousskov) Date: Tue, 25 Feb 2003 09:24:54 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Tue, 25 Feb 2003, Oskar Batuner wrote: > Abnormal termination is different from normal termination (and all > other normal messages) in the sense that it switches context from > message processing to termination and cleanup. I think this > difference justifies presence of a separate message. Both normal and abnormal termination switch callout server context from message processing to termination and cleanup, don't they? I agree that abnormal termination may have side-effects different from normal termination. For example, if the callout server caches some transaction information, it may want to purge the cache if the transaction terminated abnormally. > Another difference is that abnormal termination message may arrive > at any moment, other messages are limited by the processing context > (protocol state machine). Good point. This distinction can be useful if an implementation wants to ignore/log non-control messages sent on the control channel. > In some sense it is a question of preferences - as you have > correctly pointed out difference between normal and abnormal > termination messages can be determined by some additional reason > code in the same message. I prefer to keep an explicit difference. You have convinced me that the difference may be big enough to deserve explicit protocol support. However, since the implementation code handling each message would be almost the same, it may be more appropriate to indicate the difference via an explicit optional flag: instead of having six very similar messages: The first approach provides explicit error notification while better matching the code that is likely to be written to support the logic. It also allows us to reuse the same "error" notification mechanism with other message types if needed. Do we have an agreement now? Would you still prefer the second approach? Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Tue Feb 25 12:26:37 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA07336 for ; Tue, 25 Feb 2003 12:26:35 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1PGvuJ15383 for ietf-openproxy-bks; Tue, 25 Feb 2003 08:57:56 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1PGvsd15379 for ; Tue, 25 Feb 2003 08:57:54 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1PGvueM039793 for ; Tue, 25 Feb 2003 09:57:56 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1PGvuwY039792; Tue, 25 Feb 2003 09:57:56 -0700 (MST) (envelope-from rousskov) Date: Tue, 25 Feb 2003 09:57:56 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Tue, 25 Feb 2003, Oskar Batuner wrote: > > > In the architecture draft we are always separating OPES flow from > > > the metadata exchange. We state that metadata transfers are > > > out-of-scope for the OPES architecture. Of cause this statement does > > > not preclude using the same callout protocol - or any other - for > > > the metadata transfer. But if we permit mixing OPES data and > > > metadata (like preferences, etc.) on the same channel - well, this > > > looks at least inconsistent. > > > > I understand the intent -- keep OPES focused and small. Is there a > > precise definition of "metadata" or "OPES flow" though? For example, > > most will probably agree that service ID(s) should be attached to > > messages, but many will consider a service ID to be > > METAdata because the application protocol does not carry those. > > > > Another example: transferring a static set of preferences and rules > > should probably be out of OPES protocol scope. On the other hand, > > sending transaction-specific rules that affect, say, service > > application order may be in OPES scope. Same for specific user > > preferences and/or credentials? It seems to me that metadata specific > > to application transaction may be in scope and may be transmitted > > in-band. > > Yes, these is why I'm asking this question. We may go both ways. In > many cases it may be very natural to transfer in-band metadata > related to the current processing context. But as soon as you allow > in-band metadata transfer you may not set a clear border - what is > specific to the transaction and what is not. If we are allowed to > send user preferences when message to/from that user is processed - > why not send to remember these preferences on callout processor? And > if callout processor keeps user's data - why not to send it them > just once, in advance? We probably should not make all these choices because they are application or even environment dependent. For example, if I have a user population of 10,000 users but only 10 users can be active at a time, it may not make sense to upload the entire user database to the callout server, especially if the callout server [location] changes with time. Some metadata transfers will always be out of OPES scope. We should require support for only universally needed metathings (like service IDs). > Real decision is either yes (allow metadata transfer in-band) or no. > Metadata transfer in a limited context is not a consistent > architectural decision: all related protection measures mandated by > the architecture MUST be implemented, but if they are - why to limit > the context? Exactly! IMO, there is no way we can avoid documenting/requiring in-bound transmission of things other than application message octets. The examples above illustrate that. Thus, metadata transfers must be allowed (as a general architecture-level rule). However, we should document/require metadata transmission on as-needed basis. If later we notice a certain common pattern, then we may add a general metadata transmission message/support to the protocol. Otherwise, protocol extensions can be used for metadata transmissions that we did not need or could not predict. Either way, it is OK to violate the OPES architecture draft as long as our violations are conscious and necessary. The architecture draft can be fixed later if needed. The original question was about requiring support for special "control" channels. Those channels can be used for out-of-flow transmission of "emergency" messages such as an abnormal transaction end message. The same channel may [optionally] be used for metadata transmissions, possibly unrelated to specific transactions. However, since we do not have any specific need for such metadata transmissions, I would not care about them for now. Perhaps those transmissions will need their own channels; we will see. For now, it seems like we should add support for emergency (priority) channels (additional transport connections) as an optional optimization feature. I will make a corresponding proposal and will try to generalize the idea so that an OPES server can instruct callout server to give certain transport connections a priority, to support QoS features (e.g., application messages from Gold Members should be processed before all other application messages). Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Wed Feb 26 04:06:26 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA28701 for ; Wed, 26 Feb 2003 04:06:25 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1Q8idf05579 for ietf-openproxy-bks; Wed, 26 Feb 2003 00:44:39 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1Q8icd05569 for ; Wed, 26 Feb 2003 00:44:38 -0800 (PST) Received: from f17m-10-141.d1.club-internet.fr ([212.195.133.141] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18nxAv-0006Hp-00 for ietf-openproxy@imc.org; Wed, 26 Feb 2003 00:44:25 -0800 Message-Id: <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Thu, 27 Feb 2003 09:19:53 +0100 To: OPES WG From: jfcm Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: At 17:24 25/02/03, Alex Rousskov wrote: > > > Why not also just? I just hit escape. We may have extensions of the protocol in the future needing to add other commands. I would hate to review all the code to add etc. all the way through. jfc From owner-ietf-openproxy@mail.imc.org Wed Feb 26 04:07:47 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA28727 for ; Wed, 26 Feb 2003 04:07:44 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1Q8icK05568 for ietf-openproxy-bks; Wed, 26 Feb 2003 00:44:38 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1Q8iad05557 for ; Wed, 26 Feb 2003 00:44:36 -0800 (PST) Received: from f17m-10-141.d1.club-internet.fr ([212.195.133.141] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18nxAs-0006Hp-00 for ietf-openproxy@imc.org; Wed, 26 Feb 2003 00:44:22 -0800 Message-Id: <5.2.0.9.0.20030227081137.02c2f9a0@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Thu, 27 Feb 2003 09:12:27 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: References: <5.2.0.9.0.20030226004549.041e0d70@mail.utel.net> <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> <5.2.0.9.0.20030226004549.041e0d70@mail.utel.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: At 05:30 25/02/03, Alex Rousskov wrote: >On Wed, 26 Feb 2003, jfcm wrote: > > > No. The question was about parallel command pipe or not. My response > > was to try to see if we can have a consistent response. Obviously > > triggering a command should probably go the same way as aborting it. > >If I understand the above, that is what the current predraft proposal >does. Triggering transaction processing on the other end uses the same >mechanism as aborting that processing (an OPES message with the >corresponding transaction ID xid). Yes. And this is what I underlined in asking that command and abort be on the same pipe. > > The idea of a piped protocol being accepted, you must zap the pipe > > if you want to abort. This means a priority somewhere in the > > processus. Either in the pipe with a zapper swallowing the data. Or > > a prioritized command pipe ordering to clear the four buffers. > >Priority processing is not required to terminate a transaction. >Priority processing is an optimization -- it allows to terminate >transaction on the other side _faster_. How much faster? Depends on >how exactly one organizes priority queues and on how long those queues >are at the abortion time. > >If you recall, the predraft protocol allows for out-of-order (i.e., >prioritized) commands but does not document how the transport >connections are handled (yet). If a OPES processor opens more >connection to the callout server than there are concurrent application >messages, then a message can be sent using "spare" >[control] connections, possibly triggering a faster abort. This is the issue. You can either think of a spare connection or of a command connection. When you establish a relation between servers you may decide that conenction 0 will be the command pipe. I do prefer that approach as this is a good pipe to signal the sever is still up and running. Also, in my prefered wording, I would say that connection 0 would go to the service administrator. >I do not know what you mean by "zapper swallowing the data" in the >pipe. Can you clarify, please? Since we are talking about TCP-like >connections here, it may not be possible to "swallow" anything that >was sent. The only reliable option is to ignore messages with unknown >(e.g., already aborted) xid and/or terminate the corresponding >transport connection. Perhaps that's what you meant by swallowing? Yes and more. Even under TCP/IP you transport data in blocks/messages (IMHO due to TCP/IP there could be interest in multiplexed datagrams to go faster and get easier acks). This means that the non yet used data are in buffers and will be then processed. When I say swallowing I mean that the buffers are cleaned (and that the current processes are signaled to die). I cannot signal processes but I can clean the I/O buffers (and on the path buffers as the processes can be multi/machines). Please recall that some processes can be quite CPU consuming. Just ignoring the next protocol message may result in a lot of unecessary finishing processing. > > Then there is the message telling the server to abort and and ack to > > the requester. > >An ACK would be a waste of resources unless you want to support OPES >transaction resumption after the event of transport-layer connectivity >loss. By "resumption", I mean resuming at the place of last >synchronization, transparently to the consumer and producer. This is a good idea (this puts a process on hold or to recover from a bad situation). No, I meant that the interrupted service acks that he died. Either when dying or through a service admin obituary message. The "waste of resources" comes from the trade-off. The whole internet vision is based upon the TCP/IP trade-off you quote. The acks do exist but they are implicit and come through layer violations (the response of the application shows that it received the data). If you have a TCP/IP pipe you can use it as tansport for an higher level block transport protocol and design a extremely fast, reliable and secure value added virtual circuit switch solution. Please remember that we are in very well defined closed network. Please review my corrected draft. I kept dispatcher/services/ends blocks as equal because the command pipe issue has not been decided. But, let assume that the ONES system is built only of dispatchers and ends (and may be the ends also with a dispatcher). This means that the "dispatcher" is acting as a real time agent (like in X.400/e-mail but real time). The architecture for a virtual server would then be: ----> dispatcher -----> service admin |----> service A occurence 1 |----> service A occurence 2 |----> service B occurence 1 |----> service B occurence 2 |----> service B occurence 3 | etc. And that would be very near from the architecture I introduced in my first mail about my prototype. The inter-dispatcher protocol could be one of the services and support soap, etc. under straight TCP/IP or your OCP under a TCP/IP+ multiplexed datagram solution with a behaviour and a response time much closer of a message passing OS. This could provide a very broad set of consistent solutions? >I doubt the benefits of transaction resumption outweigh increased >protocol complexity. It seems to me that for application protocols we >know about (HTTP and SMTP), an application transaction abort or an >automated retry would be more appropriate solutions. Do you think OPES >transactions should be resumable if the transport layer fails? Let be clear. I discuss here what I name ONES of which I think OPES is a fist level. I certainly do not think that HTTP simple systems to replace a string in a page to correct an URL or to change a .jpg into a .gif needs that. But I think that an OPES may come to a point they have to behave as a ONES because of the load or of the speed. My site does not need much, but Amazone yes (we have not discussed services running in parallel and the data to keep from page to page to support a User cession?). Then SOAP is useless. In talking about ONES, we know which environement we might want to stay consistent with. Also I do not see here a clear boarder defined yet. IAB says HTTP. You add SMTP as I did. IAB says one way, yet there seem to be some real support for both ways. IAB does not say redirect and we are divided on this. Also, we may find that a more global vision may simlify and make more robust the OPES. Or extend the field of their immediate applictations. Example. I wish to have a new vision of the DNS. I send calls/mails using domain names and a 0.0.0.0 IP address that an OPES will replace with the proper current IP of the DN savnig a lot of time: no more first erroneous resolution and further redirect. And a lot of flexibility in addressing. > > The real issue is that at minimum there are four flows involved > > user/dispatcher/service/dispatcher/organizer in my own words instead > > of one. Following all the possiblities calls for an Excell > > spreadshit. > >In such cases, a higher level simplification or abstraction often >works better than enumerating endless possibilities in a table (IMO). True. See above. But to make sure about it you must be very basic first and see if the proposed abstraction resits to the list of possible cases/conflicts. We fully agree. >The proposed OCP protocol is such an abstraction (at this point in >time) because it does not really care what the consumer, producer, >source, or destinations are. True. This is why you should not embarass it with client/server issues if it is bidi. And let consider seriously an inter dispatcher network only. The disptacher as a smart plug everywhere, including on the users side (what is a firewall?) where the minimum access to a service is only a single (yes/no) rule (what is a power switcher?). >The number of flows is not important as >long as all flows are mostly independent and there is a well-known >"forking" or "initial scheduling" point (the OPES processor in our >case). True. See above. But I prefer the word "dispatcher" from the dratf. Because it is more generic and services are processed outside of the dispatcher. >The proposed protocol handles at least two flows per >application transaction and should scale well with the number of >flows. Yes. Except that these two flows can be encapsulated into a value added transport protocol for speed and flow management. That way you may have cheap and even virtual acks or service messages. Please remember that we are real time. We cannot make users wait because another user has big data to process. We must permit to optimise dispatcher and services CPU time. This certainly means priority assignement in transmitting data under the protocol. Maximum flexibility can only come from a service admin talking with the remote dispatcher. The dispatcher must then act both as a link and as transport dispatcher. jfc From owner-ietf-openproxy@mail.imc.org Wed Feb 26 04:08:24 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA28746 for ; Wed, 26 Feb 2003 04:08:22 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1Q8ieg05592 for ietf-openproxy-bks; Wed, 26 Feb 2003 00:44:40 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1Q8idd05582 for ; Wed, 26 Feb 2003 00:44:39 -0800 (PST) Received: from f17m-10-141.d1.club-internet.fr ([212.195.133.141] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18nxAw-0006Hp-00 for ietf-openproxy@imc.org; Wed, 26 Feb 2003 00:44:27 -0800 Message-Id: <5.2.0.9.0.20030227092116.02cee9a0@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Thu, 27 Feb 2003 09:24:41 +0100 To: OPES WG From: jfcm Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: At 17:57 25/02/03, Alex Rousskov wrote: >For now, it seems like we should add support for emergency (priority) >channels (additional transport connections) as an optional >optimization feature. I will make a corresponding proposal and will >try to generalize the idea so that an OPES server can instruct callout >server to give certain transport connections a priority, to support >QoS features (e.g., application messages from Gold Members should be >processed before all other application messages). This is a good point. But I would prefer the following wording and concept used/ 1. command pipe (absolute priority) as a pipe 0. Establish the relation with the service admin in my model. 2. this prevents the confusion of the mata data wording which includes both command and ancillary information. I would treat ancillary as standard transactions with the service admin subject to the general prioritization service if any. jfc From owner-ietf-openproxy@mail.imc.org Wed Feb 26 04:09:48 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA28811 for ; Wed, 26 Feb 2003 04:09:46 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1Q8iZF05547 for ietf-openproxy-bks; Wed, 26 Feb 2003 00:44:35 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1Q8iYd05533 for ; Wed, 26 Feb 2003 00:44:34 -0800 (PST) Received: from f17m-10-141.d1.club-internet.fr ([212.195.133.141] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18nxAn-0006Hp-00 for ietf-openproxy@imc.org; Wed, 26 Feb 2003 00:44:18 -0800 Message-Id: <5.2.0.9.0.20030227075118.02cedc60@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Thu, 27 Feb 2003 08:11:20 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: Fwd: RE: OPES protocol, pre-draft 01 Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="=====================_56029907==.ALT" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: --=====================_56029907==.ALT Content-Type: text/plain; charset="us-ascii"; format=flowed A A typo of mine created some confusion. Sorry. Instead of this >Service User <-----> dispatcher <----> Service Organizer > / ^ ^ \ >Other user / | | \ Other users > v v > service <--> server <----> service >controler /provider dispatcher provider >external user >/ ^ /^ >etc.. etc.. | / | > v / v > etc. / alarms,logs > > / etc > / > other ONES domains Please read that. Supervisor or Service User <-----> dispatcher <----> Service Organizer / ^ ^ \ Other user / | | \ Other users v v service <--> dispatcher <----> service controler /provider / ^ provider external user / / | / ^ etc.. etc.. / v / | / service / v dispatcher etc. / alarms,logs / etc / other ONES domains The difference is about the wrong typing "server
dispatcher" confusing with "server dispatcher". I tried to keep it denser and using "server dispatcher" was a wrong wording (server's dispatcher was intended but I thought "'s" was only for persons?). Anyway using "server" as a separate machine was wrong. Using "secondary dispatcher" (like in the DNS) would also be wrong. This only wants to show that there are thres building blccks: - dispatchers - services - ends That services are actually made of a service adminstrator triggering the services. and that the ends may be of tree different natures : - user : they call and get the service - organiser : they organize their access/responses to use the ONES (used on the flow) - supervisor : they organize their responses as a permanent cooperative strategy. This only shows that there is a simple (three building blocks, with limited number of building block types) but wich a lot more of interaction types than in the OPES draft model. My only interest right now is that we understand clearly this way what is in the OPES area (in the charter of this group) and how it may benefit from ONES analysis (up to the WG to discuss issues over OPES charter). If we do not do that we risk to confuse? jfc --=====================_56029907==.ALT Content-Type: text/html; charset="us-ascii" A A typo of mine created some confusion. Sorry. Instead of this

Service User <-----> dispatcher <----> Service Organizer
                   /  ^      ^  \     
Other user       /    |      |    \  Other users
                      v      v
                service <--> server  <----> service
controler     /provider      dispatcher     provider
external user /                  ^            /^                                         
etc.. etc..                      |          /  |
                                 v        /    v
                                etc.    /   alarms,logs                                            
                                      /     etc
                                    /
                 other ONES domains

Please read that.

                                       Supervisor or
Service User <-----> dispatcher <----> Service Organizer
                   /  ^      ^  \     
Other user        /   |      |   \  Other users
                      v      v
                 service <--> dispatcher <----> service
controler      /provider       / ^              provider
external user /               /  |             /  ^                                         
etc.. etc..                  /   v           /    |
                            /  service     /      v
                  dispatcher   etc.      /   alarms,logs                                            
                                       /     etc
                                     /
                  other ONES domains


The difference is about the wrong typing "server<br>dispatcher" confusing with "server dispatcher".

I tried to keep it denser and using "server dispatcher" was a wrong wording (server's dispatcher was intended but I thought "'s" was only for persons?). Anyway using "server" as a separate machine was wrong. Using "secondary dispatcher" (like in the DNS) would also be wrong.

This only wants to show that there are thres building blccks:

- dispatchers
- services
- ends

That services are actually made of a service adminstrator triggering the services. and that the ends may be of tree different natures :

- user : they call and get the service
- organiser : they organize their access/responses to use the ONES (used on the flow)
- supervisor : they organize their responses as a permanent cooperative strategy.

This only shows that there is a simple (three building blocks, with limited number of building block types) but wich a lot more of interaction types than in the OPES draft model.

My only interest right now is that we understand clearly this way what is in the OPES area (in the charter of this group) and how it may benefit from ONES analysis (up to the WG to discuss issues over OPES charter). If we do not do that we risk to confuse?

jfc





 

--=====================_56029907==.ALT-- From owner-ietf-openproxy@mail.imc.org Wed Feb 26 13:00:30 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA18265 for ; Wed, 26 Feb 2003 13:00:30 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QHNq714770 for ietf-openproxy-bks; Wed, 26 Feb 2003 09:23:52 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QHNpd14766 for ; Wed, 26 Feb 2003 09:23:51 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1QHNqeM074578 for ; Wed, 26 Feb 2003 10:23:52 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1QHNqa1074577; Wed, 26 Feb 2003 10:23:52 -0700 (MST) (envelope-from rousskov) Date: Wed, 26 Feb 2003 10:23:52 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: WG meeting agenda Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: My understanding is that OPES WG meeting is scheduled for Monday, March 17th, 2003: http://www.ietf.org/meetings/agenda_56.html Based on IETF documentation, the meeting date and time are now final. Please correct me if I am wrong. We should probably hash out an agenda. Markus, do you already have an agenda draft we can contribute to? Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Wed Feb 26 13:55:11 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA19711 for ; Wed, 26 Feb 2003 13:55:09 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QIBN818605 for ietf-openproxy-bks; Wed, 26 Feb 2003 10:11:23 -0800 (PST) Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QIBMd18600 for ; Wed, 26 Feb 2003 10:11:22 -0800 (PST) Received: from oskar3 (h00a0c5e5337d.ne.client2.attbi.com[24.218.184.133]) by rwcrmhc51.attbi.com (rwcrmhc51) with SMTP id <20030226181119051003ki2me>; Wed, 26 Feb 2003 18:11:19 +0000 From: "Oskar Batuner" To: "OPES WG" Subject: RE: OPES protocol, pre-draft 01 Date: Wed, 26 Feb 2003 13:11:54 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit See below. > -----Original Message----- > From: owner-ietf-openproxy@mail.imc.org > [mailto:owner-ietf-openproxy@mail.imc.org]On Behalf Of Alex Rousskov > Sent: Tuesday, February 25, 2003 11:25 AM > To: OPES WG > Subject: RE: OPES protocol, pre-draft 01 > > > > On Tue, 25 Feb 2003, Oskar Batuner wrote: > > > Abnormal termination is different from normal termination (and all > > other normal messages) in the sense that it switches context from > > message processing to termination and cleanup. I think this > > difference justifies presence of a separate message. > > Both normal and abnormal termination switch callout server context > from message processing to termination and cleanup, don't they? I > agree that abnormal termination may have side-effects different from > normal termination. For example, if the callout server caches some > transaction information, it may want to purge the cache if the > transaction terminated abnormally. > > > Another difference is that abnormal termination message may arrive > > at any moment, other messages are limited by the processing context > > (protocol state machine). > > Good point. This distinction can be useful if an implementation wants > to ignore/log non-control messages sent on the control channel. > > > In some sense it is a question of preferences - as you have > > correctly pointed out difference between normal and abnormal > > termination messages can be determined by some additional reason > > code in the same message. I prefer to keep an explicit difference. > > You have convinced me that the difference may be big enough to deserve > explicit protocol support. However, since the implementation code > handling each message would be almost the same, it may be more > appropriate to indicate the difference via an explicit optional flag: > > > > > > instead of having six very similar messages: > > > > > > > > > The first approach provides explicit error notification while better > matching the code that is likely to be written to support the logic. > It also allows us to reuse the same "error" notification mechanism > with other message types if needed. > > Do we have an agreement now? Would you still prefer the second > approach? I'd prefer something in between: -- issued by server, processing is done, "you know the results" -- issued by any party, "just forget it", results of previous processing can not be relied upon and should be discarded Oskar From owner-ietf-openproxy@mail.imc.org Wed Feb 26 13:59:20 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA19855 for ; Wed, 26 Feb 2003 13:59:19 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QIZoe19576 for ietf-openproxy-bks; Wed, 26 Feb 2003 10:35:50 -0800 (PST) Received: from server2.fastmail.fm (ny2.fastmail.fm [66.111.4.3]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QIZmd19572 for ; Wed, 26 Feb 2003 10:35:49 -0800 (PST) Received: from server2.fastmail.fm (localhost [127.0.0.1]) by fastmail.fm (Postfix) with ESMTP id 52AAA1BBEF; Wed, 26 Feb 2003 13:35:47 -0500 (EST) Received: from 127.0.0.1 ([127.0.0.1] helo=server2.fastmail.fm) by fastmail.fm with SMTP; Wed, 26 Feb 2003 13:35:47 -0500 Received: by server2.fastmail.fm (Postfix, from userid 99) id 4D98849DFB; Wed, 26 Feb 2003 13:35:47 -0500 (EST) Content-Disposition: inline Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="ISO-8859-1" MIME-Version: 1.0 X-Mailer: MIME::Lite 1.2 (F2.71; T1.001; A1.51; B2.12; Q2.03) From: "Markus Hofmann" To: "Alex Rousskov" , "OPES Group" Date: Wed, 26 Feb 2003 13:35:47 -0500 X-Epoch: 1046284547 X-Sasl-enc: Q5Lrh20WzycXhIN0EOp6Yg Subject: Re: WG meeting agenda References: In-Reply-To: Message-Id: <20030226183547.4D98849DFB@server2.fastmail.fm> Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit Alex, > We should probably hash out an agenda. Markus, do you already have an > agenda draft we can contribute to? I'm still on he road at the moment without reasonable access to email (no chance to dig through all the emails from the last week...), will be back online next week and catch up... With respect to agenda items, please send email to Marshall and myself for any suggestion, we'll try to set-up the agenda next week. Thanks, Markus From owner-ietf-openproxy@mail.imc.org Wed Feb 26 14:59:03 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA21891 for ; Wed, 26 Feb 2003 14:59:01 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QJSRW24597 for ietf-openproxy-bks; Wed, 26 Feb 2003 11:28:27 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QJSPd24593 for ; Wed, 26 Feb 2003 11:28:25 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1QJSReM078550 for ; Wed, 26 Feb 2003 12:28:27 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1QJSR1W078549; Wed, 26 Feb 2003 12:28:27 -0700 (MST) (envelope-from rousskov) Date: Wed, 26 Feb 2003 12:28:27 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Wed, 26 Feb 2003, Oskar Batuner wrote: > > You have convinced me that the difference may be big enough to deserve > > explicit protocol support. However, since the implementation code > > handling each message would be almost the same, it may be more > > appropriate to indicate the difference via an explicit optional flag: > > > > > > > > > > > > instead of having six very similar messages: > > > > > > > > > > > > > > > > > > The first approach provides explicit error notification while better > > matching the code that is likely to be written to support the logic. > > It also allows us to reuse the same "error" notification mechanism > > with other message types if needed. > > > > Do we have an agreement now? Would you still prefer the second > > approach? > > I'd prefer something in between: > > > > -- issued by server, > processing is done, > "you know the results" Great! We have an agreement on the above messages. Note that processor may need to send "normal" messages as well: this is both an application-dependent and OPES service dependent issue that we probably should not restrict too much without good reasons. > -- issued by any party, > "just forget it", > results of previous processing > can not be relied upon > and should be discarded I agree with the definition. You prefer a specific error code for this specific level of error. I prefer to keep it similar with the other abnormal messages. Since processor may need to send "normal" messages too (see above), and since the final difference between our approaches is minor (special message name versus a special message flag), I would make the changes to have all <*-end> OPES messages to use an optional "error" field for now and wait for more objections/comments. We have a semantics-level agreement. The remaining syntax-level difference may even disappear once we start selecting specific bindings and encodings for OPES. Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Wed Feb 26 15:22:18 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA22740 for ; Wed, 26 Feb 2003 15:22:16 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QJfYR25135 for ietf-openproxy-bks; Wed, 26 Feb 2003 11:41:34 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QJfXd25130 for ; Wed, 26 Feb 2003 11:41:33 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1QJfMY10689; Wed, 26 Feb 2003 14:41:23 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6YYD6>; Wed, 26 Feb 2003 14:41:22 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD75404FD7663@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , OPES WG Subject: RE: OPES protocol, pre-draft 01 Date: Wed, 26 Feb 2003 14:41:19 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2DDCF.08AC3EF0" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2DDCF.08AC3EF0 Content-Type: text/plain hi, +1 Abbie > -----Original Message----- > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > Sent: Wednesday, February 26, 2003 2:28 PM > To: OPES WG > Subject: RE: OPES protocol, pre-draft 01 > > > > On Wed, 26 Feb 2003, Oskar Batuner wrote: > > > > You have convinced me that the difference may be big enough to > > > deserve explicit protocol support. However, since the > implementation > > > code handling each message would be almost the same, it > may be more > > > appropriate to indicate the difference via an explicit optional > > > flag: > > > > > > > > > > > > > > > > > > instead of having six very similar messages: > > > > > > > > > > > > > > > > > > > > > > > > > > > The first approach provides explicit error notification > while better > > > matching the code that is likely to be written to support > the logic. > > > It also allows us to reuse the same "error" notification > mechanism > > > with other message types if needed. > > > > > > Do we have an agreement now? Would you still prefer the second > > > approach? > > > > I'd prefer something in between: > > > > > > > > -- issued by server, > > processing is done, > > "you know the results" > > Great! We have an agreement on the above messages. Note that > processor may need to send "normal" messages as > well: this is both an application-dependent and OPES service > dependent issue that we probably should not restrict too much > without good reasons. > > > -- issued by any party, > > "just forget it", > > results of previous processing > > can not be relied upon > > and should be discarded > > I agree with the definition. You prefer a specific error code > for this specific level of error. I prefer to keep it similar > with the other abnormal messages. Since processor may need to > send "normal" messages too (see above), and > since the final difference between our approaches is minor > (special message name versus a special message flag), I would > make the changes to have all <*-end> OPES messages to use an > optional "error" field for now and wait for more objections/comments. > > We have a semantics-level agreement. The remaining > syntax-level difference may even disappear once we start > selecting specific bindings and encodings for OPES. > > Thank you, > > Alex. > > ------_=_NextPart_001_01C2DDCF.08AC3EF0 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES protocol, pre-draft 01

hi,

+1

Abbie

> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> Sent: Wednesday, February 26, 2003 2:28 = PM
> To: OPES WG
> Subject: RE: OPES protocol, pre-draft 01
>
>
>
> On Wed, 26 Feb 2003, Oskar Batuner = wrote:
>
> > > You have convinced me that the = difference may be big enough to
> > > deserve explicit protocol support. = However, since the
> implementation
> > > code handling each message would be = almost the same, it
> may be more
> > > appropriate to indicate the = difference via an explicit optional
> > > flag:
> > >
> > >   <data-end xid amid [error] = reason>
> > >   <message-end xid amid = [error] reason>
> > >   <xaction-end xid [error] = reason>
> > >
> > > instead of having six very similar = messages:
> > >
> > >   <data-end-error xid amid = reason>
> > >   <message-end-error xid amid = reason>
> > >   <xaction-end-error xid = reason>
> > >   <data-end-ok xid amid = reason>
> > >   <message-end-ok xid amid = reason>
> > >   <xaction-end-ok xid = reason>
> > >
> > > The first approach provides explicit = error notification
> while better
> > > matching the code that is likely to = be written to support
> the logic.
> > > It also allows us to reuse the same = "error" notification
> mechanism
> > > with other message types if = needed.
> > >
> > > Do we have an agreement now? Would = you still prefer the second
> > > approach?
> >
> > I'd prefer something in between:
> >
> >   <data-end xid amid [error] = reason>
> >   <message-end xid amid = [error] reason>
> >   <xaction-end xid = reason>       -- issued by = server,
> = >           &n= bsp;           &n= bsp;           &n= bsp; processing is done,
> = >           &n= bsp;           &n= bsp;           &n= bsp; "you know the results"
>
> Great! We have an agreement on the above = messages. Note that
> processor may need to send "normal" = <xaction-end> messages as
> well: this is both an application-dependent and = OPES service
> dependent issue that we probably should not = restrict too much
> without good reasons.
>
> >   <xaction-abort xid = reason> -- issued by any party,
> = >           &n= bsp;           &n= bsp;         "just forget = it",
> = >           &n= bsp;           &n= bsp;         results of = previous processing
> = >           &n= bsp;           &n= bsp;         can not be relied = upon
> = >           &n= bsp;           &n= bsp;         and should be = discarded
>
> I agree with the definition. You prefer a = specific error code
> for this specific level of error. I prefer to = keep it similar
> with the other abnormal messages. Since = processor may need to
> send "normal" <xaction-end> = messages too (see above), and
> since the final difference between our = approaches is minor
> (special message name versus a special message = flag), I would
> make the changes to have all <*-end> OPES = messages to use an
> optional "error" field for now and = wait for more objections/comments.
>
> We have a semantics-level agreement. The = remaining
> syntax-level difference may even disappear once = we start
> selecting specific bindings and encodings for = OPES.
>
> Thank you,
>
> Alex.
>
>

------_=_NextPart_001_01C2DDCF.08AC3EF0-- From owner-ietf-openproxy@mail.imc.org Wed Feb 26 16:51:11 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA27787 for ; Wed, 26 Feb 2003 16:51:09 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QLTC229563 for ietf-openproxy-bks; Wed, 26 Feb 2003 13:29:12 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QLTAd29559 for ; Wed, 26 Feb 2003 13:29:10 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1QLTDeM081323 for ; Wed, 26 Feb 2003 14:29:13 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1QLTDCW081322; Wed, 26 Feb 2003 14:29:13 -0700 (MST) (envelope-from rousskov) Date: Wed, 26 Feb 2003 14:29:13 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Thu, 27 Feb 2003, jfcm wrote: > At 17:24 25/02/03, Alex Rousskov wrote: > > > > > > > > Why not also just? > What would be the meaning of the OPES message? The three old messages above are based on three persistent "objects" or "states" that OPES protocol has to maintain (data flow, application message, and application transaction). See predraft for detailed definitions. What does refer to? How is it different from, for example, ? > I just hit escape. We may have extensions of the protocol in the > future needing to add other commands. I would hate to review all the > code to add etc. all the way through. We cannot discuss or accommodate nonexistent extensions because we do not even have an extension framework yet. What can be extended? How? Without such a framework, I cannot (yet) say whether new <*-end> messages may be needed. "Hitting escape" in a user agent usually means the end of the application transaction, which is already supported with the message. OPES is not about user actions or interfaces though. It is about modifying application protocol messages. In general, if OPES protocol deals with some kind of persistency (like data flow or transaction state), there has to be a way to end that persistency. In most cases, you want the flexibility of ending one persistency without ending others that are external to it. For example, ending an application message must not end transaction processing (in general) because a transaction may include many semi-independent application messages. That is why we have the three <*-end> messages now. If we discover that there are too many persistent "things" that need to be terminated, we would have to redesign and/or, perhaps, introduce a meta-message like the one below: Alex. From owner-ietf-openproxy@mail.imc.org Wed Feb 26 17:46:43 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00385 for ; Wed, 26 Feb 2003 17:46:41 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QMJIU03274 for ietf-openproxy-bks; Wed, 26 Feb 2003 14:19:18 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QMJGd03267 for ; Wed, 26 Feb 2003 14:19:16 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1QMJJeM082462 for ; Wed, 26 Feb 2003 15:19:19 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1QMJJjJ082461; Wed, 26 Feb 2003 15:19:19 -0700 (MST) (envelope-from rousskov) Date: Wed, 26 Feb 2003 15:19:19 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: <5.2.0.9.0.20030227081137.02c2f9a0@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030226004549.041e0d70@mail.utel.net> <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> <5.2.0.9.0.20030223101644.04899da0@mail.utel.net> <5.2.0.9.0.20030226004549.041e0d70@mail.utel.net> <5.2.0.9.0.20030227081137.02c2f9a0@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Thu, 27 Feb 2003, jfcm wrote: > At 05:30 25/02/03, Alex Rousskov wrote: > >On Wed, 26 Feb 2003, jfcm wrote: > > > > > No. The question was about parallel command pipe or not. My response > > > was to try to see if we can have a consistent response. Obviously > > > triggering a command should probably go the same way as aborting it. > > > >If I understand the above, that is what the current predraft proposal > >does. Triggering transaction processing on the other end uses the same > >mechanism as aborting that processing (an OPES message with the > >corresponding transaction ID xid). > > Yes. And this is what I underlined in asking that command and > abort be on the same pipe. The location of the message (i.e., the connection it is received on) is a different story. To make aborts efficient, you may want to send an abort message on a separate pipe that is not clogged with data messages waiting to be processed (or to be discarded, actually). > This is the issue. You can either think of a spare connection or of > a command connection. When you establish a relation between servers > you may decide that conenction 0 will be the command pipe. I do > prefer that approach as this is a good pipe to signal the sever is > still up and running. Also, in my prefered wording, I would say that > connection 0 would go to the service administrator. I think we can have a more general/flexible scheme that still allows what you want: I would like OPES dispatcher to be able to give connections (and possibly transactions/messages) optional priority. If the callout server honors the priority setting, then we can have high-priority "control" connection, for example. If the callout server does not honor the priority setting or if there is no setting, then all connections SHOULD be treated with the same priority. In the latter case, a "control" connection is still possible to establish and may still be "faster" if it has fewer messages. It is relatively easy for modern implementations that manage multiple connections to support rough connection priorities. It usually boils down to polling a subset of all opened connections more often. Thus, this optional and simple optimization is likely to be supported by implementations that care about performance. As an additional optimization, we could make it possible for either side to establish a connection that only that side can send on. This optimization will take care of the situations where a callout server pollutes high-priority connection with statistics or other unimportant messages. It would be good to have a way for one side to reach the other side as fast as possible, as long as both sides play by the rules. The difficulty here is that the callout server is not allowed to open its own connections and, hence, is at the dispatcher will as far as connection optimization is concerned. > Even under TCP/IP you transport data in blocks/messages (IMHO due to > TCP/IP there could be interest in multiplexed datagrams to go faster > and get easier acks). This means that the non yet used data are in > buffers and will be then processed. When I say swallowing I mean > that the buffers are cleaned (and that the current processes are > signaled to die). I cannot signal processes but I can clean the I/O > buffers (and on the path buffers as the processes can be > multi/machines). Clearing unprocessed (by application) TCP buffers might be possible under a very limited set of conditions. For example, the buffers must be known to contain no useful messages and no partial messages. The application must have access to pretty low-level TCP controls. Also, simply handling/keeping TCP packets is already a significant performance hit and resource drain; discarding them without application-level processing does not return back wasted CPU cycles. Finally, TCP-level fiddling is possible if OPES protocol is implemented directly on top of TCP, but may not be possible for higher-level bindings. > Please recall that some processes can be quite CPU > consuming. Just ignoring the next protocol message > may result in a lot of unecessary finishing processing. True. I do not think it is avoidable under general assumptions though. See above. This is the price to pay for reliable message transmission and other nice TCP properties. Fortunately, aborts should not happen often. We optimize the common case, which is the right thing to do. > If you have a TCP/IP pipe you can use it as tansport for an higher > level block transport protocol and design a extremely fast, reliable > and secure value added virtual circuit switch solution. Please > remember that we are in very well defined closed network. Please > review my corrected draft. > > I kept dispatcher/services/ends blocks as equal because the command > pipe issue has not been decided. But, let assume that the ONES > system is built only of dispatchers and ends (and may be the ends > also with a dispatcher). This means that the "dispatcher" is acting > as a real time agent (like in X.400/e-mail but real time). > > The architecture for a virtual server would then be: > > ----> dispatcher -----> service admin > |----> service A occurence 1 > |----> service A occurence 2 > |----> service B occurence 1 > |----> service B occurence 2 > |----> service B occurence 3 > | etc. > > And that would be very near from the architecture I > introduced in my first mail about my prototype. And that is how OPES processor and callout server may be implemented! A callout server may have an "admin" module that receives all messages and directs them to appropriate services, doing load balancing if desired. > The inter-dispatcher protocol could be one of the > services and support soap, etc. under straight > TCP/IP or your OCP under a TCP/IP+ multiplexed > datagram solution with a behaviour and a response > time much closer of a message passing OS. > > This could provide a very broad set of consistent > solutions? Since "this" is actually how OPES may be implemented, I am glad you think that it could provide a very broad set of consistent solutions. > >I doubt the benefits of transaction resumption outweigh increased > >protocol complexity. It seems to me that for application protocols we > >know about (HTTP and SMTP), an application transaction abort or an > >automated retry would be more appropriate solutions. Do you think OPES > >transactions should be resumable if the transport layer fails? > > Let be clear. I discuss here what I name ONES of which I think OPES > is a fist level. I certainly do not think that HTTP simple systems > to replace a string in a page to correct an URL or to change a .jpg > into a .gif needs that. > > But I think that an OPES may come to a point they have to behave as > a ONES because of the load or of the speed. My site does not need > much, but Amazone yes (we have not discussed services running in > parallel and the data to keep from page to page to support a User > cession?). You have to be more specific for me to follow your logic. What is it exactly that Amazon needs and that OPES lacks? > Then SOAP is useless. In talking about ONES, we know which > environement we might want to stay consistent with. Also I do not > see here a clear boarder defined yet. I am not sure about SOAP, but the lack of clear boarder may be attributed to the lack of understanding what ONES is. So far, I have only seen abstract drawings that seem to match OPES architecture pretty well. Perhaps it is just me, but I still do not know what ONES is and how it is different from OPES. > Also, we may find that a more global vision may simlify and make > more robust the OPES. Or extend the field of their immediate > applictations. I agree. That is why I am trying hard to understand what ONES is and what it has that OPES lacks. > Example. I wish to have a new vision of the DNS. I send calls/mails > using domain names and a 0.0.0.0 IP address that an OPES will > replace with the proper current IP of the DN savnig a lot of time: > no more first erroneous resolution and further redirect. And a lot > of flexibility in addressing. Now we are getting somewhere. An example! However, I am not sure I fully understand the example: When I send e-mails, I use domain names, not IP addresses. The name resolution happens when it is necessary to find a mail handler for the named host. I am not sure how that can be avoided. Can you clarify how the "new DNS" would differ from the current one? How OPES would be better at resolving domain names than current gethostbyname() system calls? > ... But I prefer the word "dispatcher" from the dratf. Because it is > more generic and services are processed outside of the dispatcher. You are right. Most references to OPES processor in the current protocol predraft should be replaced with OPES dispatcher, I guess. > >The proposed protocol handles at least two flows per > >application transaction and should scale well with the number of > >flows. > > Yes. Except that these two flows can be encapsulated into a value > added transport protocol for speed and flow management. That way you > may have cheap and even virtual acks or service messages. > > Please remember that we are real time. We cannot make users wait > because another user has big data to process. We must permit to > optimise dispatcher and services CPU time. This certainly means > priority assignement in transmitting data under the protocol. > Maximum flexibility can only come from a service admin talking with > the remote dispatcher. The dispatcher must then act both as a link > and as transport dispatcher. See above for a proposal on how to add priorities to OPES connections (and, hence, possibly enable QoS discrimination you are talking about). I hope it addresses your needs. If not, please specify what "real time" needs are not addressed by the current scheme and the above priority proposal. Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Wed Feb 26 17:52:42 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00643 for ; Wed, 26 Feb 2003 17:52:40 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QMUXR03611 for ietf-openproxy-bks; Wed, 26 Feb 2003 14:30:33 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QMUVd03606 for ; Wed, 26 Feb 2003 14:30:31 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1QMUYeM082702 for ; Wed, 26 Feb 2003 15:30:34 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1QMUYfi082701; Wed, 26 Feb 2003 15:30:34 -0700 (MST) (envelope-from rousskov) Date: Wed, 26 Feb 2003 15:30:34 -0700 (MST) From: Alex Rousskov To: OPES WG Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: <5.2.0.9.0.20030227092116.02cee9a0@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030227092116.02cee9a0@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Thu, 27 Feb 2003, jfcm wrote: > At 17:57 25/02/03, Alex Rousskov wrote: > >For now, it seems like we should add support for emergency (priority) > >channels (additional transport connections) as an optional > >optimization feature. I will make a corresponding proposal and will > >try to generalize the idea so that an OPES server can instruct callout > >server to give certain transport connections a priority, to support > >QoS features (e.g., application messages from Gold Members should be > >processed before all other application messages). > > This is a good point. But I would prefer the following wording and > concept used/ > > 1. command pipe (absolute priority) as a pipe 0. Establish the > relation with the service admin in my model. > > 2. this prevents the confusion of the mata data wording which > includes both command and ancillary information. I would treat > ancillary as standard transactions with the service admin subject to > the general prioritization service if any. My proposal allows creation of a connection with priority higher than all other priorities. That connection can be used by OPES dispatcher and service admin as the "command pipe" that you propose in item #1 above. It would be up to OPES dispatcher what to send on that "command pipe". Your implementation, for example, may avoid sending metadata (auxiliary) messages on the command pipe if desired. Do you agree that the priority-based scheme I proposed both addresses your needs and is more general/flexible? Or does it lack some necessary features? Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Wed Feb 26 18:13:06 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA02227 for ; Wed, 26 Feb 2003 18:13:04 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QMoYK04219 for ietf-openproxy-bks; Wed, 26 Feb 2003 14:50:34 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QMoWd04215 for ; Wed, 26 Feb 2003 14:50:32 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1QMoZeM083155 for ; Wed, 26 Feb 2003 15:50:35 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1QMoZ7a083154; Wed, 26 Feb 2003 15:50:35 -0700 (MST) (envelope-from rousskov) Date: Wed, 26 Feb 2003 15:50:35 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: <5.2.0.9.0.20030227075118.02cedc60@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030227075118.02cedc60@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Thu, 27 Feb 2003, jfcm wrote: > Please read that. > > Supervisor or > Service User <-----> dispatcher <----> Service Organizer > / ^ ^ \ > Other user / | | \ Other users > v v > service <--> dispatcher <----> service > controler /provider / ^ provider > external user > / / | / ^ > etc.. etc.. / v / | > / service / v > dispatcher etc. / alarms,logs > > / etc > / > other ONES domains > > This only wants to show that there are thres building blccks: > > - dispatchers > - services > - ends OPES includes dispatchers and services and excludes ends (beyond application protocols the ends use). If you claim the above is different from OPES at the building blocks level, then you need to show that your ends need to communicate with dispatchers using ONES protocols (not application protocols). I think you are doing that later, where you tell us that "ends" are not necessarily "users" or "content producers". > That services are actually made of a service adminstrator triggering > the services. Possible under OPES. > and that the ends may be of tree different natures : > > - user : they call and get the service Using an application protocol, I presume (not OPES/ONES protocol)? Or do you expect a browser (for example) to be ONES-aware? In what way? > - organiser : they organize their access/responses to use the ONES > (used on the flow) I do not understand this part. Organizers organize user messages? How is this different from OPES dispatcher or processor role? > - supervisor : they organize their responses as a permanent > cooperative strategy. I do not understand this part. Can you be more specific? What is a permanent cooperative strategy? Does "their" refer to users? How is this different from OPES dispatcher or processor role? Also, where is a content producer (e.g., origin server)? Is that also a service in ONES terminology? I would expect it to be one of the "ends" since content producers are not ONES-aware. > This only shows that there is a simple (three building blocks, with > limited number of building block types) but wich a lot more of > interaction types than in the OPES draft model. I do not see more interaction yet. Perhaps that is because I do not understand what "organiser" and "supervisor" are. All other entities seem to match OPES entities well. > My only interest right now is that we understand clearly this way > what is in the OPES area (in the charter of this group) and how it > may benefit from ONES analysis (up to the WG to discuss issues over > OPES charter). Same here! I think we are getting closer. Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Wed Feb 26 18:16:50 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA02303 for ; Wed, 26 Feb 2003 18:16:48 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QMrI604300 for ietf-openproxy-bks; Wed, 26 Feb 2003 14:53:18 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QMrHd04296 for ; Wed, 26 Feb 2003 14:53:17 -0800 (PST) Received: from f13m-8-178.d1.club-internet.fr ([212.195.83.178] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18oAQQ-0006ty-00 for ietf-openproxy@imc.org; Wed, 26 Feb 2003 14:53:18 -0800 Message-Id: <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Wed, 26 Feb 2003 23:48:50 +0100 To: From: jfcm Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: References: <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Alex, we are talking about abort. And saving CPU time. And cleaning the mess possibly on may machines and pipes. At 22:29 26/02/03, Alex Rousskov wrote: >On Thu, 27 Feb 2003, jfcm wrote: > > At 17:24 25/02/03, Alex Rousskov wrote: > > > > > > > > > > > > > Why not also just? > > > >What would be the meaning of the OPES message? The three old >messages above are based on three persistent "objects" or "states" >that OPES protocol has to maintain (data flow, application message, and >application transaction). See predraft for detailed definitions. What >does refer to? How is it different from, for example, >? as I say would be for the three of them altoegether. I want to get rid of everything about that transaction. I do not want to consider anything left which may be ressource consuming or unclean anywhere. > > I just hit escape. We may have extensions of the protocol in the > > future needing to add other commands. I would hate to review all the > > code to add etc. all the way through. > >We cannot discuss or accommodate nonexistent extensions because we do >not even have an extension framework yet. Well this will just introduce it :-). What I mean is that meaning "to kill immediately whatever is related to the transaction", will also kill whatever extension we may add in the future. >What can be extended? How? Without such a framework, I cannot (yet) say >whether new <*-end> >messages may be needed. Right. But I do not want that. means that everything related to that transaction "never existed". You must understand that if you start considering messages and logger for every transaction of this protocol, in some kind of use, the protocol may be larger than the data, and disk access to take more of the machine time. >"Hitting escape" in a user agent usually means the end of the >application transaction, which is already supported with the > message. I said "also". Instead of your three messages, one zapper. >OPES is not about user actions or interfaces >though. It is about modifying application protocol messages. (1) it was an image. (2) that has direct implication on the protocol. Let say that I call a web site through a translation OPES. If it is too long to wait or if text comes by parts, I want to be able to escape. This information is to hit the translation CPU asap. As a translation operator I want to save every CPU time I can to provide the fasted and smoothest service. And if it is paying not to be cheated. >In general, if OPES protocol deals with some kind of persistency (like >data flow or transaction state), there has to be a way to end that >persistency. In most cases, you want the flexibility of ending one >persistency without ending others that are external to it. For example, >ending an application message must not end transaction processing (in >general) because a transaction may include many semi-independent >application messages. That is why we have the three <*-end> messages >now. Certainly. But here is not the case we discuss. >If we discover that there are too many persistent "things" that need to >be terminated, we would have to redesign and/or, perhaps, introduce a >meta-message like the one below: > > "what"? BTW: we are in real-time multi-processor environement. If you want to keep track of all the "things" you may have to terminate at a given time without having any zombie, you may have to have a much more complex system. Also, the servers must know they face an emergency situation because they may have to react differently from a normal termination. They do not expect any emergency in a standard . jfc From owner-ietf-openproxy@mail.imc.org Wed Feb 26 19:05:37 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA03378 for ; Wed, 26 Feb 2003 19:05:36 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1QNhNl05550 for ietf-openproxy-bks; Wed, 26 Feb 2003 15:43:23 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1QNhMd05546 for ; Wed, 26 Feb 2003 15:43:22 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1QNhPeM084406 for ; Wed, 26 Feb 2003 16:43:25 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1QNhPxT084405; Wed, 26 Feb 2003 16:43:25 -0700 (MST) (envelope-from rousskov) Date: Wed, 26 Feb 2003 16:43:25 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Wed, 26 Feb 2003, jfcm wrote: > At 22:29 26/02/03, Alex Rousskov wrote: > >On Thu, 27 Feb 2003, jfcm wrote: > > > At 17:24 25/02/03, Alex Rousskov wrote: > > > > > > > > > > > > > > > > > > Why not also just? > > > > > > >What would be the meaning of the OPES message? The three old > >messages above are based on three persistent "objects" or "states" > >that OPES protocol has to maintain (data flow, application message, and > >application transaction). See predraft for detailed definitions. What > >does refer to? How is it different from, for example, > >? > > as I say would be for the three of them altoegether. I want to > get rid of everything about that transaction. I do not want to > consider anything left which may be ressource consuming or unclean > anywhere. This is exactly what does. > Well this will just introduce it :-). What I mean is that meaning > "to kill immediately whatever is related to the transaction", will also > kill whatever extension we may add in the future. This is exactly what does. > I said "also". Instead of your three messages, one zapper. implies the other two, so only one message needs to be sent, just like with . See examples in the predraft. > Let say that I call a web site through a translation OPES. If it is > too long to wait or if text comes by parts, I want to be able to > escape. This information is to hit the translation CPU asap. As a > translation operator I want to save every CPU time I can to provide > the fasted and smoothest service. And if it is paying not to be > cheated. Looks like already supports this. > Also, the servers must know they face an emergency situation because > they may have to react differently from a normal termination. They > do not expect any emergency in a standard . That is exactly what Oskar Batuner pointed out after predraft 00 was released. The problem is now fixed -- a special "error" flag has been added to distinguish normal termination from an abort/emergency. is not a "normal" termination. Do you agree that is equivalent to ? Or is there more functionality in that is missing? Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Thu Feb 27 08:46:01 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA02301 for ; Thu, 27 Feb 2003 08:45:59 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1RDP5P05366 for ietf-openproxy-bks; Thu, 27 Feb 2003 05:25:05 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1RDP3Y05358 for ; Thu, 27 Feb 2003 05:25:04 -0800 (PST) Received: from f15v-1-223.d1.club-internet.fr ([212.195.192.223] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18oO20-0004rj-00; Thu, 27 Feb 2003 05:25:01 -0800 Message-Id: <5.2.0.9.0.20030227140122.030030d0@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Thu, 27 Feb 2003 14:08:22 +0100 To: Alex Rousskov , ietf-openproxy@imc.org From: jfcm Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: References: <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: At 00:43 27/02/03, Alex Rousskov wrote: >Do you agree that is equivalent to ? Or is >there more functionality in that is missing The more you add to the more you will make it looking it alike at a given time. But the more you add "error" the more you add possible error types confusion. Let settles that means and let developpers use it. IMHO thei will result in being first tested and being the default in the action-end case, freely tested as per the developper gusto or priorities. Also calls of reread/maintenance of the code when the protocol is enhanced and options added. not. You will see that "nr zap" will complexify over time, while will stay an immediate clean and reset command. Are you ok with that? Anyway in my version there will be support :-) Thank you. jfc From owner-ietf-openproxy@mail.imc.org Thu Feb 27 12:12:59 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA12027 for ; Thu, 27 Feb 2003 12:12:58 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1RGjv419248 for ietf-openproxy-bks; Thu, 27 Feb 2003 08:45:57 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1RGjtY19244 for ; Thu, 27 Feb 2003 08:45:55 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1RGjveM008136 for ; Thu, 27 Feb 2003 09:45:57 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1RGjv80008135; Thu, 27 Feb 2003 09:45:57 -0700 (MST) (envelope-from rousskov) Date: Thu, 27 Feb 2003 09:45:57 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: <5.2.0.9.0.20030227140122.030030d0@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> <5.2.0.9.0.20030227140122.030030d0@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: jfc, It looks like you are still missing the point: There is absolutely NO semantical difference between or or . Any compliant implementation will have exactly the same code to handle any of the three messages. We cannot argue about incompliant implementations, of course. I already explained why I would prefer not to add [up to three] more message types to OPES (which is what solution requires). We can do that later if needed. The remaining difference is only in visual formatting in the predraft document: versus . I am surprised you want to spend time arguing about this difference, especially since the OPES binding and message encoding may make the visual difference disappear. If you insist, I can say that "zap" name is inferior to "error" because "zap" is an action and "error" is information. You may have noticed that the protocol is being designed so that each side provides information about its state. The other side will interpret that information according to protocol rules and its own state. Using action names like "zap" or "terminate" or "do-what-i-tell-you" is not the best approach here because it is too specific and may not match the other side capabilities or state. The simplest example is when the other side already zapped the transaction in question. It cannot zap it again, yet the message explicitly tells it to do so. From design point of view, this is bad because it creates an exceptional situation -- the side has been asked to do something that it cannot. Note that if the current design philosophy is followed, the problem does not exist -- the side is informed that there was a fatal error on the other side, and is free to take _any_ OPES-compliant action (which, in this case, is ignoring the message). Alex. P.S. I do not know what "nr" means in your message. On Thu, 27 Feb 2003, jfcm wrote: > At 00:43 27/02/03, Alex Rousskov wrote: > >Do you agree that is equivalent to ? Or is > >there more functionality in that is missing > > The more you add to the more you will make it looking > it alike at a given time. But the more you add "error" the more you add > possible error types confusion. > > Let settles that means and let > developpers use it. IMHO thei will result in being first tested and > being the default in the action-end case, > freely tested as per the developper gusto or priorities. > > Also calls of reread/maintenance of the code when the > protocol is enhanced and options added. not. > > You will see that "nr zap" will complexify over time, while will stay > an immediate clean and reset command. Are you ok with that? > > Anyway in my version there will be support :-) > Thank you. > jfc From owner-ietf-openproxy@mail.imc.org Thu Feb 27 13:40:52 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15836 for ; Thu, 27 Feb 2003 13:40:50 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1RI8s124519 for ietf-openproxy-bks; Thu, 27 Feb 2003 10:08:54 -0800 (PST) Received: from hosting.altserver.com (hosting.altserver.com [209.124.80.2]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1RI8rY24515 for ; Thu, 27 Feb 2003 10:08:53 -0800 (PST) Received: from f09m-1-5.d1.club-internet.fr ([213.44.216.5] helo=mine.utel.net) by hosting.altserver.com with esmtp (Exim 3.36 #1) id 18oSSj-0001Zk-00 for ietf-openproxy@imc.org; Thu, 27 Feb 2003 10:08:54 -0800 Message-Id: <5.2.0.9.0.20030227184614.02ac3730@mail.utel.net> X-Sender: info+utel.net@mail.utel.net X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Thu, 27 Feb 2003 19:14:59 +0100 To: ietf-openproxy@imc.org From: jfcm Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: References: <5.2.0.9.0.20030227140122.030030d0@mail.utel.net> <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> <5.2.0.9.0.20030227140122.030030d0@mail.utel.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - hosting.altserver.com X-AntiAbuse: Original Domain - imc.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - utel.net Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: At 17:45 27/02/03, Alex Rousskov wrote: >jfc, >It looks like you are still missing the point: I think you also does, due to my incapacity to explain it in your own protocol words. So, let stop this for the time being. We will keep it mind. What I know is as a developper in my specific environment I needed it and I suppose that in the project which takes shape we will use it. But much to early. Also you may be right about it being ill construed. The development I refer to is now 15/7 years old, initially very rustic and empirical. I just know what I did with it :-) > The simplest example is when the other side already zapped the > transaction in question. It cannot zap it again, yet the message > explicitly tells it to do so. hmmmm. Sorry I lead you to a wrong understanding. My use/understanding of zap is initial pipe oriented. Clean the "thing" tree from that pipe. Using 100 times zap is totally acceptable. Actually this is the best way to synchronize. zap means swallow everything related to this pipe, clean it. Nothing in there, what ever it was. Clean buffers. Signal attached tasks to die propagate along the pipes all over the network. Or the equivalent (I use the image of my old QNX based system). May be I was wrong in assuming that it corresponded to your commands. Exemple of dispatcher rule: if (ball==zap) { send_zap(piper); clean_reset(); exit(); } So for example zap 0 would kill the network and restart the network establishment negociations, etc. Next to power down it is the "reboot" of the network. Is this clearer? jfc From owner-ietf-openproxy@mail.imc.org Thu Feb 27 14:14:28 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA17515 for ; Thu, 27 Feb 2003 14:14:27 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1RIoLu28453 for ietf-openproxy-bks; Thu, 27 Feb 2003 10:50:21 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1RIoJY28446 for ; Thu, 27 Feb 2003 10:50:19 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1RIoLeM011022 for ; Thu, 27 Feb 2003 11:50:21 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1RIoLS4011021; Thu, 27 Feb 2003 11:50:21 -0700 (MST) (envelope-from rousskov) Date: Thu, 27 Feb 2003 11:50:21 -0700 (MST) From: Alex Rousskov To: OPES Group Subject: too fast out-of-order messages Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: No optimization comes for free. Here is a problem with supporting priority message processing in OPES. I am seeking the group opinion on how the problem should be handled. We want to allow out-of-order transaction-dependent messages to be able to abort an in-progress transaction as fast as possible. This optimization has a problem: it is possible that the "fast" message will be processed by the callout server _before_ the server processes the corresponding message. If the current protocol rules are followed, that fast message will be ignored because it will not match any known transaction. Moreover, the transaction may not be terminated at all, until the timeout happens (which is exactly the opposite of what we want!). Possible sane solutions: 0) Prohibit out-of-order messages and related optimizations. 1) Prohibit out-of-order messages until the other side acknowledges that it received the message (and, hence, the transaction is now known to that other side). The acknowledgment may be done explicitly by immediately sending an message or implicitly by sending other transaction-dependent messages as a part of normal operation. We may even require immediate ACKs. 2) Require message to always be sent via the usual [slow] channel, even if another copy of the message was sent via a "fast" channel. This way, if fast message is too fast, we have no optimization but preserve the same semantics. Same carbon-copy rule can be applied to all future out-of-order messages. Note that it is OK to send two messages because the second one will be ignored by the recipient. Personally, I favor (2) because it allows for an optimization, is simple, has no overheads for those that do not support the optimization, and does not require creation of the "other side heard us" state. Note that we can inform implementors that they may _want_ to send explicit ACKs as an optional feature (they are allowed to do that already), especially if the callout server does not expect to send any other transaction-dependent messages for some time. Again, the ACKs should use existing messages. Comments/opinions? Thanks, Alex. From owner-ietf-openproxy@mail.imc.org Thu Feb 27 14:34:23 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA18329 for ; Thu, 27 Feb 2003 14:34:22 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1RJEa301123 for ietf-openproxy-bks; Thu, 27 Feb 2003 11:14:36 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1RJEYY01119 for ; Thu, 27 Feb 2003 11:14:34 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1RJEaeM012549 for ; Thu, 27 Feb 2003 12:14:36 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1RJEa56012548; Thu, 27 Feb 2003 12:14:36 -0700 (MST) (envelope-from rousskov) Date: Thu, 27 Feb 2003 12:14:36 -0700 (MST) From: Alex Rousskov To: ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft 01 In-Reply-To: <5.2.0.9.0.20030227184614.02ac3730@mail.utel.net> Message-ID: References: <5.2.0.9.0.20030227140122.030030d0@mail.utel.net> <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030227091650.02cf6bb0@mail.utel.net> <5.2.0.9.0.20030226232923.03103e30@mail.utel.net> <5.2.0.9.0.20030227140122.030030d0@mail.utel.net> <5.2.0.9.0.20030227184614.02ac3730@mail.utel.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Thu, 27 Feb 2003, jfcm wrote: > My use/understanding of zap is initial pipe oriented. Clean the > "thing" tree from that pipe. > > Using 100 times zap is totally acceptable. Actually this is the best > way to synchronize. zap means swallow everything related to this > pipe, clean it. Nothing in there, what ever it was. Clean buffers. > Signal attached tasks to die propagate along the pipes all over the > network. Or the equivalent (I use the image of my old QNX based > system). May be I was wrong in assuming that it corresponded to your > commands. There are no connection-oriented commands in the predraft yet! What you want is entirely different from a message. You want to kill every transaction that is associated with a given OPES connection. You want message that has no xid (because it applies to all transactions on the corresponding connection). is an optimization. It is semantically equivalent (almost) to sending multiple messages, but is smaller. I will add to the TODO list. We need to decide what to do with OPES connections in general before we can zap them. Moreover, you want to clear all unprocessed connection buffers which should be a different command because it has a different semantics. All implementations can support message and only very few (or none, depending on allowed OPES bindings and message encodings) can support buffer flushing. For example, if OPES messages are encoded in raw XML, then you may not be able to flush buffers because you may create partial messages and may not be able to find a beginning of the message after that. IMO, this buffer-flushing command is too low-level for OPES. Simply closing and re-opening the connection is probably more appropriate and is not much more expensive. I suggest we wait for other opinions before proceeding with this. > So for example zap 0 would kill the network and restart the network > establishment negociations, etc. Next to power down it is the > "reboot" of the network. This is even more than flushing buffers. You want to close and reopen connections as well. I think this should wait until we add connection management framework to the protocol. We need to decide whether and how OPES connections can be reopened and what effect a suddenly closed connection has on the other end. Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Thu Feb 27 16:36:08 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA24909 for ; Thu, 27 Feb 2003 16:36:07 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1RLGw805952 for ietf-openproxy-bks; Thu, 27 Feb 2003 13:16:58 -0800 (PST) Received: from sccrmhc03.attbi.com (sccrmhc03.attbi.com [204.127.202.63]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1RLGsY05948 for ; Thu, 27 Feb 2003 13:16:54 -0800 (PST) Received: from oskar3 (h00a0c5e5337d.ne.client2.attbi.com[24.218.184.133]) by sccrmhc03.attbi.com (sccrmhc03) with SMTP id <20030227211650003009s7i5e>; Thu, 27 Feb 2003 21:16:50 +0000 From: "Oskar Batuner" To: "Alex Rousskov" , "OPES Group" Subject: RE: too fast out-of-order messages Date: Thu, 27 Feb 2003 16:17:25 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 In-Reply-To: Importance: Normal Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 7bit > We want to allow out-of-order transaction-dependent messages to be > able to abort an in-progress transaction as fast as possible. There always may be a need to abort a transaction in progress, by why "out-of-order" and why "as fast as possible"? I'd strongly suggest to avoid (to prohibit) cross-channel messages. All transaction processing should be done within the channel where it has started. Some kind of message may come at any moment as one side of transaction gets an abnormal condition that makes further processing unnecessary. But there is no "urgency" in transaction termination - just no need to continue. I can propose two type of scenarios: A. callout processor gets unrecoverable error condition (type 500) in situation when OPES processor keeps a copy of initial message waiting for the next more-data request. B. Data consumer unexpectedly closes connection to the OPES server while request processing is in process. In both cases other side of transaction does not expect transaction-end message - this is why I propose to have a special message that can appear at any moment (related to protocol state machine). But there is no need to expedite transaction termination, this is just a way out of intermediate processing status. Can you propose scenarios where out-of-band message gives significant advantages? > -----Original Message----- > From: owner-ietf-openproxy@mail.imc.org > [mailto:owner-ietf-openproxy@mail.imc.org]On Behalf Of Alex Rousskov > Sent: Thursday, February 27, 2003 1:50 PM > To: OPES Group > Subject: too fast out-of-order messages > > > > > No optimization comes for free. Here is a problem with supporting > priority message processing in OPES. I am seeking the group opinion on > how the problem should be handled. > > We want to allow out-of-order transaction-dependent messages to be > able to abort an in-progress transaction as fast as possible. This > optimization has a problem: it is possible that the "fast" > message will be processed by the callout server _before_ > the server processes the corresponding message. If the > current protocol rules are followed, that fast message > will be ignored because it will not match any known transaction. > Moreover, the transaction may not be terminated at all, until the > timeout happens (which is exactly the opposite of what we want!). > > Possible sane solutions: > > 0) Prohibit out-of-order messages and related > optimizations. > > 1) Prohibit out-of-order messages until the other > side acknowledges that it received the > message (and, hence, the > transaction is now known to that other side). > The acknowledgment may be done explicitly > by immediately sending an message > or implicitly by sending other transaction-dependent > messages as a part of normal operation. > We may even require immediate ACKs. > > 2) Require message to always be sent > via the usual [slow] channel, even if another > copy of the message was sent via a "fast" channel. > This way, if fast message is too fast, we have > no optimization but preserve the same semantics. > Same carbon-copy rule can be applied to all > future out-of-order messages. Note that it is > OK to send two messages because > the second one will be ignored by the recipient. > > Personally, I favor (2) because it allows for an optimization, is > simple, has no overheads for those that do not support the > optimization, and does not require creation of the "other side heard > us" state. I support your choice (2), even in more strict definition: all transaction processing goes through the same channel on which it was started. (0) is not an option: there always are situations when regular processing can not be continued, so there should be some way out. (1) can not happen if everything goes on the same channel (in fact it is a good illustration what kind of problems may be created by asynchronous multi-channel processing). > > Note that we can inform implementors that they may _want_ to send > explicit ACKs as an optional feature (they are allowed to do that > already), especially if the callout server does not expect to send any > other transaction-dependent messages for some time. Again, the ACKs > should use existing messages. a) I think this is an overkill; b) reuse of a message in a different sematic context is usually a very bad thing. In this situation it is very easy to misinterpret keep-alive message as a confirmation (well, you may introduce transaction id in keep-alive, but IMHO this creates even more mess). Oskar From owner-ietf-openproxy@mail.imc.org Thu Feb 27 17:35:57 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA26590 for ; Thu, 27 Feb 2003 17:35:55 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1RME0p08908 for ietf-openproxy-bks; Thu, 27 Feb 2003 14:14:00 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1RMDwY08904 for ; Thu, 27 Feb 2003 14:13:58 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1RME1eM016716 for ; Thu, 27 Feb 2003 15:14:01 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1RME1gP016715; Thu, 27 Feb 2003 15:14:01 -0700 (MST) (envelope-from rousskov) Date: Thu, 27 Feb 2003 15:14:01 -0700 (MST) From: Alex Rousskov To: OPES Group Subject: RE: too fast out-of-SENT-order messages In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Thu, 27 Feb 2003, Oskar Batuner wrote: > > We want to allow out-of-order transaction-dependent messages to be > > able to abort an in-progress transaction as fast as possible. > > There always may be a need to abort a transaction in progress, > by why "out-of-order" and why "as fast as possible"? ... because some may want this optimization. For example, consider this situation: Callout server processing of each response chunk is long and "expensive". The callout server is processing the first chunk, and 9 next chunks are still in its TCP buffers waiting to be parsed and processed. The user aborts its request. There is no reason for the callout server to continue, but it will not know until it gets a message. Without the optimization in question, the callout server will not know until all 10 chunks are processed. Please note that I am just explaining the motivation, not arguing whether this optimization is worth supporting. > I'd strongly suggest to avoid (to prohibit) cross-channel messages. > All transaction processing should be done within the channel where > it has started. Some kind of message may come at > any moment as one side of transaction gets an abnormal condition > that makes further processing unnecessary. But there is no "urgency" > in transaction termination - just no need to continue. There is no semantics urgency indeed. It is a performance optimization. See above for a typical example. > I can propose two type of scenarios: A. callout processor gets > unrecoverable error condition (type 500) in situation when OPES > processor keeps a copy of initial message waiting for the next > more-data request. B. Data consumer unexpectedly closes connection > to the OPES server while request processing is in process. In > both cases other side of transaction does not expect transaction-end > message - this is why I propose to have a special message that > can appear at any moment (related to protocol state machine). Yes, your examples are valid. We already agreed to add the "abort" mechanism. The only question is whether it makes sense to allow the abort to happen faster, as a performance/resource optimization. > But there is no need to expedite transaction termination, this > is just a way out of intermediate processing status. > > Can you propose scenarios where out-of-band message > gives significant advantages? Yes, see my example above. A general optimization area is "slow and expensive processing of data chunks for messages with multiple chunks". Is this area large enough to be worth caring for? > > Possible sane solutions: > > > > 0) Prohibit out-of-order messages and related > > optimizations. > > > > 1) Prohibit out-of-order messages until the other > > side acknowledges that it received the > > message (and, hence, the > > transaction is now known to that other side). > > The acknowledgment may be done explicitly > > by immediately sending an message > > or implicitly by sending other transaction-dependent > > messages as a part of normal operation. > > We may even require immediate ACKs. > > > > 2) Require message to always be sent > > via the usual [slow] channel, even if another > > copy of the message was sent via a "fast" channel. > > This way, if fast message is too fast, we have > > no optimization but preserve the same semantics. > > Same carbon-copy rule can be applied to all > > future out-of-order messages. Note that it is > > OK to send two messages because > > the second one will be ignored by the recipient. > > > > Personally, I favor (2) because it allows for an optimization, is > > simple, has no overheads for those that do not support the > > optimization, and does not require creation of the "other side heard > > us" state. > > I support your choice (2), even in more strict definition: > all transaction processing goes through the same channel on > which it was started. That is option, (0) actually. No optimization (see below). > (0) is not an option: there always are situations when regular > processing can not be continued, so there should be some way out. Another terminology conflict! By out-of-order I mean out-of-SENT-order not out-of-semantics-order: the message that was sent later was received first. As agreed, can be sent at any time and is never out of semantics order (i.e., you can stop at any time, just like you wanted). > (1) can not happen if everything goes on the same channel (in fact > it is a good illustration what kind of problems may be created by > asynchronous multi-channel processing). Exactly. We need to decide whether the associated problems are worth the optimization benefit. BTW, if you can think of any other problems, please let me know. Keep in mind that only messages that can be sent at any time (like your abort message) will be allowed on priority channels if multi-channel support is added. We are not talking about sending data out of order, for example. > b) reuse of a message in a different sematic context is > usually a very bad thing. I agree, but the semantic context here is the same: "I am here, processing xid". > In this situation it is very easy to misinterpret keep-alive message > as a confirmation (well, you may introduce transaction id in > keep-alive, but IMHO this creates even more mess). Actually, predraft-01 does have xids and even amids in messages :-). I do not see how they can hurt, and I can give examples where they help (you may want to abort an application message due to a timeout but still be able to finish the corresponding transaction involving the other 9 messages; that is, you may want to be more specific). But this is not related to the primary question of this thread: Should we allow fast abort optimization? If yes, how to make it safe? Your last comments make the questions broader: Should we allow multi-channel transaction processing? If yes, should we allow fast abort optimization? If yes, how to make it safe? Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Fri Feb 28 05:24:55 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA19521 for ; Fri, 28 Feb 2003 05:24:54 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1SA58b25921 for ietf-openproxy-bks; Fri, 28 Feb 2003 02:05:08 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1SA55Y25914 for ; Fri, 28 Feb 2003 02:05:06 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id 6R53ZWSF; 28 Feb 2003 11:04:55 +0100 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Subject: RE: too fast out-of-SENT-order messages Date: Fri, 28 Feb 2003 11:01:49 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C135@hermes.webwasher.com> Thread-Topic: too fast out-of-SENT-order messages Thread-Index: AcLerp8/39jVTM0HQpSOD43xPH4elAAXbTTQ From: "Martin Stecher" To: "Alex Rousskov" , "OPES Group" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1SA57Y25915 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit Hi, > > > > We want to allow out-of-order transaction-dependent messages to be > > > able to abort an in-progress transaction as fast as possible. > > > > There always may be a need to abort a transaction in progress, > > by why "out-of-order" and why "as fast as possible"? > > ... because some may want this optimization. > > For example, consider this situation: Callout server processing of > each response chunk is long and "expensive". The callout server is > processing the first chunk, and 9 next chunks are still in its TCP > buffers waiting to be parsed and processed. The user aborts its > request. There is no reason for the callout server to continue, but it > will not know until it gets a message. Without the > optimization in question, the callout server will not know until all > 10 chunks are processed. > Wait a second, to avoid a misunderstanding. The example you are describing is this really using a message to abort? I thought it should actually be a message instead of because other application messages within this transaction are not affected, are they? I am somehow undecided between solutions 0) and 2). Maybe it makes sense to go for 2) and make clear in the protocol that the abort messages MUST be sent via the normal channel. It could then be an option or even extension that is not handled in the protocol from the beginning to send these messages in copy on a fast channel. Regards Martin From owner-ietf-openproxy@mail.imc.org Fri Feb 28 13:33:09 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA08360 for ; Fri, 28 Feb 2003 13:33:07 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1SI5rA27311 for ietf-openproxy-bks; Fri, 28 Feb 2003 10:05:53 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1SI5pY27302 for ; Fri, 28 Feb 2003 10:05:51 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1SI5reM045240 for ; Fri, 28 Feb 2003 11:05:53 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1SI5rqB045239; Fri, 28 Feb 2003 11:05:53 -0700 (MST) (envelope-from rousskov) Date: Fri, 28 Feb 2003 11:05:53 -0700 (MST) From: Alex Rousskov To: OPES Group Subject: RE: too fast out-of-SENT-order messages In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C135@hermes.webwasher.com> Message-ID: References: <72992B39BBD9294BB636A960E89AE02E50C135@hermes.webwasher.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Fri, 28 Feb 2003, Martin Stecher wrote: > > For example, consider this situation: Callout server processing of > > each response chunk is long and "expensive". The callout server is > > processing the first chunk, and 9 next chunks are still in its TCP > > buffers waiting to be parsed and processed. The user aborts its > > request. There is no reason for the callout server to continue, but it > > will not know until it gets a message. Without the > > optimization in question, the callout server will not know until all > > 10 chunks are processed. > > Wait a second, to avoid a misunderstanding. The example you are > describing is this really using a message to abort? I > thought it should actually be a message > instead of because other application > messages within this transaction are not affected, are they? You may be right, depending on the application protocol and state. For example, it makes no sense to continue processing an _uncachable_ HTTP response if the client has left and the whole transaction (the HTTP request-response pair) can be terminated. If response is cachable, it may make sense to continue, especially if a large portion of the response has been received already, and if the cache stores post-processed responses. I doubt OPES can require usage of a particular -end message because optimal behavior would depend on application protocol and state. We can only recommend that the implementations on both sides of the protocol leave as much decision freedom for the other side as possible; if it makes sense to send a message-abort instead of a transaction-abort, the former message should be sent. Here is a related question: should we let the other side affect the abort decision on OPES level? Perhaps the callout server is doing some logging or accounting and MUST see every byte received by the OPES dispatcher, even if a part of the transaction is aborted by the dispatcher. Should we add some kind of message? Or should we assume that the dispatcher always knows callout server needs and vice versa? > I am somehow undecided between solutions 0) and 2). > Maybe it makes sense to go for 2) and make clear in the protocol > that the abort messages MUST be sent via the normal channel. > It could then be an option or even extension that is not handled > in the protocol from the beginning to send these messages in > copy on a fast channel. Yes, that is how (2) should be documented/implemented. It is an optional (MAY) optimization feature for the sender. I think it MUST be supported by the recipient though to make optimization decisions simpler. In general, I lean towards allowing out-of-sent-order messages because without them the protocol is too slow/inefficient for handling large transfers. I think we ought to allow more interactivity between callout server and the dispatcher because such interactivity may be the only way to implement certain adaptation-specific features. We just need to make this optimization simple and safe (and solution (2) may achieve that). Here is a motivational example. Imagine and OPES system that modifies certain rare URLs (links) in HTML responses based on a large set of rules. The rules tell what URLs should be rewritten and how. Maybe this is a CDN or another kind of a site mirror, does not matter. The rules are maintained at the OPES dispatcher side (for reasons outside of our control). The rule set is too large to be transferred to the callout server. Thus, when the callout server detects a candidate URL, it has to ask the dispatcher whether this URL needs modifications and what modifications it needs. This can probably be done via some OPES extensions. If these request/response pairs are sent via normal "data" channels, the processing may be very slow and require a lot of outstanding state. If the request/response pairs can go on a separate connection, the problem goes away; the delay to process each rare URL does not depend on the length of the message (or the total length of outstanding chunks, to be precise). The question is, of course, whether we should care about such situations or declare them out of OPES scope instead. We can say, for example, that if an OPES implementation needs interactive dialog between OPES agents, that dialog should be supported on separate non-OPES channels. I am not sure this would be the right solution because the dialog will probably be related to OPES communications and, hence, can benefit from being within the OPES framework. Does anybody have good examples where interactivity among OPES agents would be required and would benefit from being performed over OPES channels? Thank you, Alex. From owner-ietf-openproxy@mail.imc.org Fri Feb 28 13:50:31 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA08758 for ; Fri, 28 Feb 2003 13:50:29 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1SILpm28490 for ietf-openproxy-bks; Fri, 28 Feb 2003 10:21:51 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1SILoY28486 for ; Fri, 28 Feb 2003 10:21:50 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1SILdP09355; Fri, 28 Feb 2003 13:21:39 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6Z5K3>; Fri, 28 Feb 2003 13:21:39 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD7540502AB11@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , OPES Group Subject: RE: too fast out-of-order messages Date: Fri, 28 Feb 2003 13:21:37 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2DF56.3B756108" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2DF56.3B756108 Content-Type: text/plain > -----Original Message----- > From: > Sent: Friday, February 28, 2003 1:15 PM > To: 'Alex Rousskov'; OPES Group > Subject: RE: too fast out-of-order messages > > > > see inline, > > abbie > > > -----Original Message----- > > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > > Sent: Thursday, February 27, 2003 1:50 PM > > To: OPES Group > > Subject: too fast out-of-order messages > > > > > > > SNIP > > > Possible sane solutions: > > > > 0) Prohibit out-of-order messages and related > > optimizations. > > > > 1) Prohibit out-of-order messages until the other > > side acknowledges that it received the > > message (and, hence, the > > transaction is now known to that other side). > > The acknowledgment may be done explicitly > > by immediately sending an message > > or implicitly by sending other transaction-dependent > > messages as a part of normal operation. > > We may even require immediate ACKs. > > > > 2) Require message to always be sent > > via the usual [slow] channel, even if another > > copy of the message was sent via a "fast" channel. > > This way, if fast message is too fast, we have > > no optimization but preserve the same semantics. > > Same carbon-copy rule can be applied to all > > future out-of-order messages. Note that it is > > OK to send two messages because > > the second one will be ignored by the recipient. > > > > Personally, I favor (2) because it allows for an > > optimization, is simple, has no overheads for those that do > > not support the optimization, and does not require creation > > of the "other side heard us" state. > > > > Note that we can inform implementors that they may _want_ to > > send explicit ACKs as an optional feature (they are allowed > > to do that already), especially if the callout server does > > not expect to send any other transaction-dependent messages > > for some time. Again, the ACKs should use existing > > messages. > > > > Comments/opinions? > > > > SNIP > > Personally, I would prefere option 0 for now (simply for > protocol simplicity reasons). > However, I do see your reasoning for option 2. > > Abbie > > > > > > > > > > > ------_=_NextPart_001_01C2DF56.3B756108 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: too fast out-of-order messages

> -----Original Message-----
> From:
> Sent: Friday, February 28, 2003 1:15 PM
> To: 'Alex Rousskov'; OPES Group
> Subject: RE: too fast out-of-order = messages
>
>
>
> see inline,
>
> abbie
>
> > -----Original Message-----
> > From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> > Sent: Thursday, February 27, 2003 1:50 = PM
> > To: OPES Group
> > Subject: too fast out-of-order = messages
> >
> >
> >
> SNIP
>
> > Possible sane solutions:
> >
> >     0) Prohibit = out-of-order messages and related
> >        = optimizations.
> >
> >     1) Prohibit = out-of-order messages until the other
> >        side = acknowledges that it received the
> >        = <xaction-start> message (and, hence, the
> >        = transaction is now known to that other side).
> >        The = acknowledgment may be done explicitly
> >        by = immediately sending an <i-am-here> message
> >        or = implicitly by sending other transaction-dependent
> >        messages = as a part of normal operation.
> >        We may = even require immediate ACKs.
> >
> >     2) Require = <xaction-end> message to always be sent
> >        via the = usual [slow] channel, even if another
> >        copy of = the message was sent via a "fast" channel.
> >        This way, = if fast message is too fast, we have
> >        no = optimization but preserve the same semantics.
> >        Same = carbon-copy rule can be applied to all
> >        future = out-of-order messages. Note that it is
> >        OK to send = two <xaction-end> messages because
> >        the second = one will be ignored by the recipient.
> >
> > Personally, I favor (2) because it allows = for an
> > optimization, is simple, has no overheads = for those that do
> > not support the optimization, and does not = require creation
> > of the "other side heard us" = state.
> >
> > Note that we can inform implementors that = they may _want_ to
> > send explicit ACKs as an optional feature = (they are allowed
> > to do that already), especially if the = callout server does
> > not expect to send any other = transaction-dependent messages
> > for some time. Again, the ACKs should use = existing
> > <i-am-here> messages.
> >
> > Comments/opinions?
> >
>
> SNIP
>
> Personally, I would prefere option 0 for now = (simply for
> protocol simplicity reasons).
> However, I do see your reasoning for option = 2.
>
> Abbie
>
>
>
>
>
>
>
>
>
>
>

------_=_NextPart_001_01C2DF56.3B756108-- From owner-ietf-openproxy@mail.imc.org Fri Feb 28 13:52:09 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA08786 for ; Fri, 28 Feb 2003 13:52:06 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1SIMRI28516 for ietf-openproxy-bks; Fri, 28 Feb 2003 10:22:27 -0800 (PST) Received: from zcars04f.nortelnetworks.com (zcars04f.nortelnetworks.com [47.129.242.57]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1SIMQY28511 for ; Fri, 28 Feb 2003 10:22:26 -0800 (PST) Received: from zcard309.ca.nortel.com (zcard309.ca.nortel.com [47.129.242.69]) by zcars04f.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id h1SIMBP09409; Fri, 28 Feb 2003 13:22:11 -0500 (EST) Received: by zcard309.ca.nortel.com with Internet Mail Service (5.5.2653.19) id <17C6Z5L1>; Fri, 28 Feb 2003 13:22:12 -0500 Message-ID: <87609AFB433BD5118D5E0002A52CD7540502AB13@zcard0k6.ca.nortel.com> From: "Abbie Barbir" To: Alex Rousskov , ietf-openproxy@imc.org Subject: RE: OPES protocol, pre-draft 01 Date: Fri, 28 Feb 2003 13:22:11 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C2DF56.4F638960" Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C2DF56.4F638960 Content-Type: text/plain > -----Original Message----- > From: > Sent: Friday, February 28, 2003 1:08 PM > To: 'Alex Rousskov'; ietf-openproxy@imc.org > Subject: RE: OPES protocol, pre-draft 01 > > > > See inline, > > Abbie > > > -----Original Message----- > > From: Alex Rousskov [mailto:rousskov@measurement-factory.com] > > Sent: Thursday, February 27, 2003 2:15 PM > > To: ietf-openproxy@imc.org > > Subject: RE: OPES protocol, pre-draft 01 > > > > > > > > On Thu, 27 Feb 2003, jfcm wrote: > > > > > My use/understanding of zap is initial pipe oriented. Clean the > > > "thing" tree from that pipe. > > > > > > Using 100 times zap is totally acceptable. Actually this is > > the best > > > way to synchronize. zap means swallow everything related to > > this pipe, > > > clean it. Nothing in there, what ever it was. Clean > buffers. Signal > > > attached tasks to die propagate along the pipes all over > > the network. > > > Or the equivalent (I use the image of my old QNX based > > system). May be > > > I was wrong in assuming that it corresponded to your commands. > > > > There are no connection-oriented commands in the predraft yet! > > --- Abbie, that is correct. > > > > > What you want is entirely different from a > > message. You want to kill every transaction that is > > associated with a given OPES connection. You want > > message that has no xid (because it > > applies to all transactions on the corresponding connection). > > > > is an optimization. It is semantically equivalent > > (almost) to sending multiple messages, but > > is smaller. I will add to the TODO list. > > We need to decide what to do with OPES connections in general > > before we can zap them. > > > > ---ABbie > In TODO list is a good place for it. > > > > Moreover, you want to clear all unprocessed connection > > buffers which should be a different command because it has a > > different semantics. All implementations can support > > message and only very few (or none, > > depending on allowed OPES bindings and message > > encodings) can support buffer flushing. For example, if OPES > > messages are encoded in raw XML, then you may not be able to > > flush buffers because you may create partial messages and may > > not be able to find a beginning of the message after that. > > > > IMO, this buffer-flushing command is too low-level for OPES. > > Simply closing and re-opening the connection is probably more > > appropriate and is not much more expensive. I suggest we wait > > for other opinions before proceeding with this. > > > > ---Abbie, ALex, Amen, let us keep the vry low level out. > > > > So for example zap 0 would kill the network and restart > the network > > > establishment negociations, etc. Next to power down it is > > the "reboot" > > > of the network. > > > > --- Abbie, Yuk...... > > > This is even more than flushing buffers. You want to close > > and reopen connections as well. I think this should wait > > until we add connection management framework to the protocol. > > We need to decide whether and how OPES connections can be > > reopened and what effect a suddenly closed connection has on > > the other end. > > > > Thank you, > > > > Alex. > > > > ---- Agree > > > ------_=_NextPart_001_01C2DF56.4F638960 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: OPES protocol, pre-draft 01

> -----Original Message-----
> From:
> Sent: Friday, February 28, 2003 1:08 PM
> To: 'Alex Rousskov'; = ietf-openproxy@imc.org
> Subject: RE: OPES protocol, pre-draft 01
>
>
>
> See inline,
>
> Abbie
>
> > -----Original Message-----
> > From: Alex Rousskov [mailto:rousskov@measure= ment-factory.com]
> > Sent: Thursday, February 27, 2003 2:15 = PM
> > To: ietf-openproxy@imc.org
> > Subject: RE: OPES protocol, pre-draft = 01
> >
> >
> >
> > On Thu, 27 Feb 2003, jfcm wrote:
> >
> > > My use/understanding of zap is = initial pipe oriented. Clean the
> > > "thing" tree from that = pipe.
> > >
> > > Using 100 times zap is totally = acceptable. Actually this is
> > the best
> > > way to synchronize. zap means swallow = everything related to
> > this pipe,
> > > clean it. Nothing in there, what ever = it was. Clean
> buffers. Signal
> > > attached tasks to die propagate along = the pipes all over
> > the network.
> > > Or the equivalent (I use the image of = my old QNX based
> > system). May be
> > > I was wrong in assuming that it = corresponded to your commands.
> >
> > There are no connection-oriented commands = in the predraft yet!
>
> --- Abbie, that is correct.
>
> >
> > What you want is entirely different from a = <xaction-end>
> > message. You want to kill every = transaction that is
> > associated with a given OPES connection. = You want
> > <transactionS-end ...> message that = has no xid (because it
> > applies to all transactions on the = corresponding connection).
> >
> > <transactions-end> is an = optimization. It is semantically equivalent
> > (almost) to sending multiple = <transaction-end> messages, but
> > is smaller. I will add = <transactions-end> to the TODO list.
> > We need to decide what to do with OPES = connections in general
> > before we can zap them.
> >
>
> ---ABbie
> In TODO list is a good place for it.
> >
> > Moreover, you want to clear all = unprocessed connection
> > buffers which should be a different = command because it has a
> > different semantics. All implementations = can support
> > <transactions-end> message and only = very few (or none,
> > depending on allowed OPES bindings and = message
> > encodings) can support buffer flushing. = For example, if OPES
> > messages are encoded in raw XML, then you = may not be able to
> > flush buffers because you may create = partial messages and may
> > not be able to find a beginning of the = message after that.
> >
> > IMO, this buffer-flushing command is too = low-level for OPES.
> > Simply closing and re-opening the = connection is probably more
> > appropriate and is not much more = expensive. I suggest we wait
> > for other opinions before proceeding with = this.
> >
>
> ---Abbie, ALex, Amen, let us keep the vry low = level out.
>
> > > So for example zap 0 would kill the = network and restart
> the network
> > > establishment negociations, etc. Next = to power down it is
> > the "reboot"
> > > of the network.
> >
>
> --- Abbie, Yuk......
>
> > This is even more than flushing buffers. = You want to close
> > and reopen connections as well. I think = this should wait
> > until we add connection management = framework to the protocol.
> > We need to decide whether and how OPES = connections can be
> > reopened and what effect a suddenly closed = connection has on
> > the other end.
> >
> > Thank you,
> >
> > Alex.
> >
>
> ---- Agree
> >
>

------_=_NextPart_001_01C2DF56.4F638960-- From owner-ietf-openproxy@mail.imc.org Fri Feb 28 15:41:46 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA12616 for ; Fri, 28 Feb 2003 15:41:44 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1SKG6T07701 for ietf-openproxy-bks; Fri, 28 Feb 2003 12:16:06 -0800 (PST) Received: from wwsmtp.webwasher.com ([217.146.159.49]) by above.proper.com (8.11.6/8.11.3) with SMTP id h1SKG3Y07697 for ; Fri, 28 Feb 2003 12:16:04 -0800 (PST) Received: from hermes.webwasher.com [192.168.1.3] id KPGLQRIY; 28 Feb 2003 21:16:05 +0100 content-class: urn:content-classes:message Subject: AW: too fast out-of-SENT-order messages MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Date: Fri, 28 Feb 2003 21:12:56 +0100 Message-ID: <72992B39BBD9294BB636A960E89AE02E50C136@hermes.webwasher.com> Thread-Topic: too fast out-of-SENT-order messages Thread-Index: AcLfVXao6+XzyEwYTGupPmSbxXekqAAD3OXA From: "Martin Stecher" To: "Alex Rousskov" , "OPES Group" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by above.proper.com id h1SKG5Y07698 Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: Content-Transfer-Encoding: 8bit > > > > For example, consider this situation: Callout server processing of > > > each response chunk is long and "expensive". The callout server is > > > processing the first chunk, and 9 next chunks are still in its TCP > > > buffers waiting to be parsed and processed. The user aborts its > > > request. There is no reason for the callout server to > continue, but it > > > will not know until it gets a message. Without the > > > optimization in question, the callout server will not > know until all > > > 10 chunks are processed. > > > > Wait a second, to avoid a misunderstanding. The example you are > > describing is this really using a message to abort? I > > thought it should actually be a message > > instead of because other application > > messages within this transaction are not affected, are they? > > You may be right, depending on the application protocol and state. For > example, it makes no sense to continue processing an _uncachable_ HTTP > response if the client has left and the whole transaction (the HTTP > request-response pair) can be terminated. If response is cachable, it > may make sense to continue, especially if a large portion of the > response has been received already, and if the cache stores > post-processed responses. > > [...] It seems that I misunderstood the whole concept. I thought a transaction could be more than a single request-response pair. I thought when the OPES processor opens a transaction it tells the callout server what to do with all following application messages, e.g. following hundreds of HTTP requests before the transaction stops. That would have been a nice way to reduce the exchange of always the same meta data. Martin From owner-ietf-openproxy@mail.imc.org Fri Feb 28 16:03:41 2003 Received: from above.proper.com (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA13298 for ; Fri, 28 Feb 2003 16:03:40 -0500 (EST) Received: (from majordomo@localhost) by above.proper.com (8.11.6/8.11.3) id h1SKcpM08348 for ietf-openproxy-bks; Fri, 28 Feb 2003 12:38:51 -0800 (PST) Received: from measurement-factory.com (measurement-factory.com [206.168.0.5]) by above.proper.com (8.11.6/8.11.3) with ESMTP id h1SKcnY08344 for ; Fri, 28 Feb 2003 12:38:50 -0800 (PST) Received: from measurement-factory.com (localhost [127.0.0.1]) by measurement-factory.com (8.12.6/8.12.5) with ESMTP id h1SKcpeM049908 for ; Fri, 28 Feb 2003 13:38:52 -0700 (MST) (envelope-from rousskov@measurement-factory.com) Received: (from rousskov@localhost) by measurement-factory.com (8.12.6/8.12.5/Submit) id h1SKcp4p049907; Fri, 28 Feb 2003 13:38:51 -0700 (MST) (envelope-from rousskov) Date: Fri, 28 Feb 2003 13:38:51 -0700 (MST) From: Alex Rousskov To: OPES Group Subject: Re: AW: too fast out-of-SENT-order messages In-Reply-To: <72992B39BBD9294BB636A960E89AE02E50C136@hermes.webwasher.com> Message-ID: References: <72992B39BBD9294BB636A960E89AE02E50C136@hermes.webwasher.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-ietf-openproxy@mail.imc.org Precedence: bulk List-Archive: List-Unsubscribe: List-ID: On Fri, 28 Feb 2003, Martin Stecher wrote: > It seems that I misunderstood the whole concept. I am glad you asked these questions, then! Clearly, predraft needs to do a better job at defining OPES messages. > I thought a transaction could be more than a single request-response > pair. It could be, depending on the application protocol. This is an _application_ transaction we are talking about here. For HTTP, an application transaction could be a (request, response) pair or it might be all messages related to the same (persistent) client connection, or server connection (less likely). Note that some protocols (e.g., HTTP) do not define a clear transaction boundaries. In those cases, it will be always up to the OPES dispatcher what to call an application transaction, I guess. > I thought when the OPES processor opens a transaction it tells the > callout server what to do with all following application messages, > e.g. > following hundreds of HTTP requests before the transaction stops. > That would have been a nice way to reduce the exchange of always the > same meta data. You seem to be thinking about OPES transactions here. Those are not (yet?) defined in predraft version of the protocol. We need application transactions and application messages as OPES "objects" because a lot of processing state (metadata) will be associated with them. We may also need OPES transactions because, like you said, there may be lots of state associated with them as well. We already have OPES messages, of course. I was thinking that OPES transactions might correspond to OPES connections. That is why I did not introduce them into the draft yet, waiting to hear more opinions on the connection management issues. It is probably about time for me to make a specific proposal though. In your experience, what kind of metadata/state would be associated with an OPES transaction (separate from the application transaction)? Same service IDs? Dispatcher- and callout-server-specific info? Anything else? Thank you, Alex. P.S. Perhaps we should change "xid" to "axid" to emphasize that it identifies an Application transaction.