Mediated Heuristic Approaches and Alternative Social Welfare Definitions for Complex Contract Negotiations Involving Highly Uncorrelated Utility Spaces

Automated negotiation provides an important mechanism to reach agreements among distributed decision makers (Beer et al., 1999; Kraus et al., 1998; Lai et al., 2004; Rosenschein & Zlotkin, 1994). It has been extensively studied from the perspective of e-commerce (Guttman et al., 1998; He et al., 2003; Lopez-Carmona et al., 2006; Sierra, 2004), though it can be seen from a more general perspective as a paradigm to solve coordination and cooperation problems in complex systems (Jennings, 2001; Klein et al., 2002), providing a mechanism for autonomous agents to reach agreements on, e.g., task allocation, resource sharing, or surplus division (Fatima et al., 2004; Kersten & Noronha, 1998; Zhang et al., 2005).

the negotiators should be considered as an integral part within the multiparty negotiation protocols.
In addition, in the last years there has been an increasing interest in complex negotiations (Klein et al., 2003;Lopez-Carmona et al., 2011). Complexity of a negotiation scenario may depend on several factors, like the cardinality of the solution space, the number of negotiating agents, the number of issues under negotiation, the degree of interdependency between the issues, and structural properties of the different agent preference landscapes, like ruggedness, modality or correlation length (Vassilev et al., 2003). Specially challenging are those scenarios involving highly rugged or highly auto-uncorrelated utility spaces, since traditional negotiation approaches (mostly intended for linear or quasi-concave utility functions) cannot be applied effectively to these scenarios.
We can find some successful research works in the literature addressing negotiation in complex utility spaces (Fatima et al., 2009;Hindriks et al., 2006;Klein et al., 2003). However, most of these approaches rely on monotonic utility spaces, binary valued issues, low-order dependencies, a fixed set of defined apriorisolutions or simplification of the negotiation space, and there are very few works addressing auto-uncorrelated utility spaces with high-order issue interdependencies. One of the most relevant approaches in this field is (Ito et al., 2007). They propose a bidding mechanism, which is based on taking random samples of the contract space and applying simulated annealing to these samples to identify high utility regions for each agent, sending these regions as bids to a mediator, and then performing a search in the mediator to find overlaps between the bids of the different agents. Experiments show that this approach achieves high effectiveness (measured as high optimality rates and low failure rates for the negotiations) in the evaluation scenario they describe. However, this approach performs worse as the circumstances of the scenario turn harder (that is, when the utility functions present low autocorrelation). Furthermore, the bidding-based negotiation protocol presents some scalability concerns when the number of agents increases. Under these circumstances, the failure rate increases drastically, raising the need for an alternative approach.
Therefore, new approaches are needed if automated negotiation is to be applied to settings involving highly uncorrelated preference spaces. Here the use of heuristic optimization techniques to assist negotiation processes has been revealed as a promising approach. In fact, negotiation scenarios and optimization problems are often closely related, since there are many similarities in the ways both problem families are defined and addressed. For example, negotiating agents are usually utility optimizers, and negotiation mechanisms are often evaluated in terms of their ability to reach Pareto-optimal solutions. Multi-objective optimization has been widely used for negotiation support (Wierzbicki et al., 1993), and negotiation mechanisms have also been used to solve multiobjective optimization problems, usually by distributing the different objectives among negotiating agents (Shi et al., 2006). Therefore, some of the techniques studied in multiobjective optimization may be used in negotiation, and vice versa.
Apart from using optimization techniques, there is a trend in automated complex negotiation to use mediated protocols. The existence of a mediator in the negotiation allows agents to provide more information to facilitate a successful outcome for the negotiation without revealing too much information to their counterparts. Furthermore, mediated protocols allow to decouple agents' individual objectives (normally, to maximize their profit) from the social objective of maximizing social welfare.
In this chapter, we will discuss the application of nonlinear optimization techniques to mediated negotiations involving highly uncorrelated utility spaces, and the effect of the different notions of social welfare in this kind of negotiations.

Multi-attribute negotiation in complex utility spaces
Multi-attribute negotiation may be seen as an interaction between two or more agents with the goal of reaching an agreement about a range of issues which usually involves solving a conflict of interests between the agents. This kind of interaction has been widely studied in different research areas, such as game theory (Nash, 1950;Rosenschein & Zlotkin, 1994), distributed artificial intelligence (Ehtamo et al., 1999;Faratin et al., 1998;Gatti & Amigoni, 2005;Ito et al., 2008;Klein et al., 2003;Lai et al., 2006) and economics (Raiffa, 1982).
Though there has been an increasing interest in complex negotiations in the last years, little efforts have been made to study complexity itself within negotiation (apart from computational complexity, which has been thoroughly studied in many scenarios). Therefore, if we want to be able to assess complexity in negotiations, we need to resort to other knowledge areas. An area where many authors have dealt with complexity characterization and measurement is optimization. In fact, negotiation scenarios and optimization problems are often closely related, since there are many similarities in the ways both problem families are defined and addressed (Sawaragi et al., 1985;Shi et al., 2006;Wierzbicki et al., 1993). Therefore, some of the concepts studied in multiobjective optimization may be used in negotiation.
In the context of a multi-attribute negotiation, complexity may depend at least on the number of issues, the level of interdependency between the preferences on the issues, the size of the issue domain, and the possibility of change over time of the negotiation context. The method to describe preferences and the structural properties of the agents' utility spaces also play a fundamental role in the complexity of the negotiation scenario. This is specially true when optimization techniques are used to find high utility regions within an agent's utility space, or to find deals among different agents. A constraint-based preference space, for instance, may present discontinuities which make gradient based optimizers not applicable, while differentiable utility functions contribute to a faster local optimization.
Therefore, to study complexity in negotiation scenarios, we may find useful to characterize structural complexity of the agents' utility spaces, and to this end we may benefit from existing research on function characterization for optimization. In particular, some of the results in the analysis of fitness landscapes in evolutionary computation are specially useful, since they identify a set of landscape structural properties, such as modality (Horn & Goldberg, 1994), ruggedness, smoothness and neutrality (Vassilev et al., 2003), which are also interesting regarding search complexity within the solution space in a negotiation scenario. Furthermore, there are some techniques which allow to measure the complexity of a given space. Most of the approaches we can find in the literature are based on the correlation between different samples of the fitness function, like fitness distance correlation metrics (Tomassini et al., 2005), or stochastic models representing the correlation structure of the space (Hordijk, 1995). A metric which is easy to compute in most scenarios and allows to make quantitative evaluations about the complexity of a fitness or utility landscape is correlation length or correlation distance.
Correlation distance is defined as the minimum distance ψ which makes correlation fall below a given threshold (usually 0.5). This measure gives an idea of the distance we can move throughout the solution space while keeping a certain correlation between samples (Manderick et al., 1991). Throughout this work, we will use the terms "correlated" or "uncorrelated" to refer to utility spaces in terms of their correlation lengths. (Klein et al., 2003) present the first negotiation protocols specific for complex preference spaces. They propose a simulated annealing-based approach, a refined version based on a parity-maintaining annealing mediator, and an unmediated version of the negotiation protocol. Of great interest in this work are the positive results about the use of simulated annealing as a way to regulate agent decision making, along with the use of an expressive bid language to allow the mediator to improve its proposals. However, this expressiveness is somewhat limited, with only four possible valuations which allow the mediator to decide which contract to use as a parent for mutation, but not in which direction to mutate it. On the other hand, the performed experiments only consider the bilateral negotiation scenario, though authors claim that the multiparty generalization is simple. Finally, the family of negotiation protocols they propose are specific for binary issues and binary dependencies. Higher-order dependencies and continuous-valued issues, common in many real-world contexts, are known to generate more challenging utility landscapes which are not considered in their work. (Luo et al., 2003) propose a fuzzy constraint based framework for multi-attribute negotiations. In this framework a buyer agent defines a set of fuzzy constraints to describe its preferences. The proposals of the buyer agent are a set of hard constraints which are extracted from the set of fuzzy constraints. The seller agent responds with an offer or with a relaxation request. The buyer then decides whether to accept or reject an offer, or to relax some constraints by priority from the lowest to highest. In 2007) an improvement to Luo's model is presented. They devise an expressive negotiation protocol where proposals include a valuation of the different constraints, and seller's responses may contain explicit relaxation requests. It means that a seller agent may suggest the specific relaxation of one or more constraints. The relaxation suggested by a seller agent is based on utility and viability criteria, which improves the negotiation process. Though these constraint-based works model discontinuous preference spaces, the operators used to compute utility and the utility spaces defined yield monotonic preference spaces, which are far from the complex preference spaces covered in our work.
Another interesting approach to solve the computational cost and complexity of negotiating interdependent issues is to simplify the negotiation space. (Hindriks et al., 2006) propose a weighted approximation technique to simplify the utility space. They show that for smooth utility functions the application of this technique results in an outcome that closely matches the outcome based on the original interdependent utility structure. The method is evaluated for a number of randomly generated utility spaces with interdependent issues. Experiments show that this approach can achieve reasonably good outcomes for utility spaces with simple dependencies. However, an approximation error that deviates negotiation outcomes from the optimal solutions cannot be avoided, and this error may become larger when the approximated utility functions become more complex. Authors acknowledge as a necessary future work to study which kind of functions can be approximated accurately enough using this mechanism. Another limitation of this approach is that it is necessary to estimate a region of the utility space where the actual outcome is expected to be (i.e. it is assumed that the region is known aprioriby the agents).
In (Robu et al., 2005) utility graphs are used to model issue interdependencies for binary-valued issues. Utility graphs are inspired by graph theory and probabilistic influence networks to derive efficient heuristics for non-mediated bilateral negotiations about multiple issues. The idea is to decompose highly non-linear utility functions in sub-utilities of clusters of inter-related items. They show how utility graphs can be used to model an opponent's preferences. In this approach agents need prior information about the maximal structure of the utility space to be explored. Authors argue that this prior information could be obtained through a history of past negotiations or the input of domain experts. However, our approach has the advantage that outcomes can be reached without any prior information and that it is not restricted to binary-valued issues.
There are several proposals which employ genetic algorithms to learn opponent's preferences according to the history of the counter-offers based upon stochastic approximation. In (Choi et al., 2001) a system based on genetic-algorithms for electronic business is proposed. In this work the utility functions are restricted to take a product combination form (i.e. utility of an outcome is the product of the utility values of the different issues). The objective function used is based on the comparison of the changes of consecutive offers. Small changes of an issue suggest that this issue is more important. For each new population, the protocol enforces that the generated candidates cannot be better than the previous offer. Unlike other negotiation models based on genetic algorithms, this proposal adapts to the environment by dynamically modifying its mutation rate. (Lau et al., 2004) have also reported a negotiation mechanism for non-mediated automated negotiations based on genetic algorithms. The fitness function relies on three aspects: an agent's own preference, the distance of a candidate offer from the previous opponent's offer, and time pressure. In this work agents' preferences are quantified by a linear aggregation of the issue valuations. However, rugged and discontinuous preference spaces are not explored. In (Chou et al., 2007) a genetic algorithm is proposed which is based on a joint elitism operation and a joint fitness operation. In the joint elitism operation an agent stores the latest offers received from the opponent. The joint fitness operation combines agent's own utility function and euclidean distance to the opponent's offer. In this work two different negotiation scenarios are considered. In the first one utility is defined as the weighted sum of the different issue values (i.e. issues are independent). The second scenario defines a utility function where there is a master issue and a set of slave issues. Utility is calculated as the weighted sum of the different issue values, but the weights of the slave and master issues change according to the value of the master issue.
In (Yager, 2007) a mediated negotiation framework for multi-agent negotiation is presented. This framework involves a mediation step in which the individual preference functions are aggregated to obtain a group preference function. The main interest is focused on the implementation of the mediation rule where they allow a linguistic description using fuzzy logic. A notable feature of their approach is the inclusion of a mechanism rewarding the agents for being open to alternatives other than simply their most preferred. The negotiation space and utility values are assumed to be arbitrary (i.e. preferences can be uncorrelated). However, the set of possible solutions is defined aprioriand is fixed. Moreover, the preference function needs to be provided to the mediation step in the negotiation process, and pareto-optimality is not considered. Instead, a stopping rule is considered, which determines when the rounds of mediation stop. (Fatima et al., 2009) analyze bilateral multi-issue negotiation involving nonlinear utility functions. They consider the case where issues are divisible and there are time constraints in the form of deadlines and discounts. They show that it is possible to reach Pareto-optimal agreements by negotiating all the issues together, and that finding an equilibrium is not computationally easy if the agents' utility functions are nonlinear. In order to overcome this complexity they investigate two solutions: approximating nonlinear utilities with linear ones; and using a simultaneous procedure where the issues are discussed in parallel but independently of each other. This study shows that the equilibrium can be computed in polynomial time. An important part of this work is the complexity analysis and estimated approximation error analysis performed over the proposed approximated equilibrium strategies. Heuristic approaches have generally the drawback of the lack of a solid mathematical structure which guarantees their viability, which raises the need of an exhaustive experimental evaluation. An adequate complexity analysis and establishing a bound over the approximation error contribute to give heuristic approaches part of the technical soundness they usually lack. Among the limitations of the proposal, we can point out that this work is focused on symmetric agents where the preferences are distributed identically, and the utility functions are separable in nonlinear polynomials of a single variable. This somewhat limits the complexity of the preference space.
Finally, combinatorial auctions (Giovannucci et al., 2010;Hunsberger & Grosz, 2000;Sakurai et al., 2000;Sandholm, 2002;Volgenant, 2002;Xia et al., 2005) can enable large-scale collective decision making in nonlinear domains, but only of a very limited type (i.e. negotiations consisting solely of resource allocation decisions). Multi-attribute auctions, wherein buyers advertise their utility functions, and sellers compete to offer the highest-utility bid (Bichler et al., 1999;Parkes & Kalagnanam, 2005;Teich et al., 2006) are also aimed at a fundamentally limited problem (a purchase negotiation with a single buyer) and require full revelation of preference information.
In summary, in the existing research nearly all the models which assume issue interdependency rely on monotonic utility spaces, binary valued issues, low-order dependencies, or a fixed set of defined ap r i o r isolutions. Simplification of the negotiation space has also been reported as a valid approach for simple utility functions, but it cannot be used with higher-order issue dependencies which make utility spaces very uncorrelated. Therefore, new approaches are needed if automated negotiation is to be applied to settings involving highly uncorrelated preference spaces. One of the most promising avenues for research in this area is the use of mediated protocols, and the use of heuristics based on optimization mechanisms to make these protocols converge to a suitable agreement. We will see in the following some of these approaches, with special emphasis in the way they use alternative social welfare notions to impact the final outcome of the negotiations.

An auction-based approach for negotiations in complex, constraint-based utility spaces
In this section we describe a negotiation model specifically designed for complex utility spaces, which was first described in . The model relies on the use of a constraint-based agent preference structure, a mediated, auction-based interaction protocol capable to support multiple iterations and expressive feedback at each iteration, and a set of heuristic mechanisms for the negotiating agents and the mediator. For the purpose of this chapter, we will focus on the iterative negotiation protocol and the mechanisms used for deal identification at the mediator.

Constraint-based nonlinear utility spaces
Nonlinear agent preferences can be described by using different categories of functions, like K-additive utility functions (Chevaleyre et al., 2004), bidding languages (Nisan, 2006), or weighted constraints (Ito et al., 2008). In this section we focus on nonlinear utility spaces generated by means of weighted constraints. In these cases, agents' utility functions are described by defining a set of constraints. Each constraint represents a region with one or more dimensions, and has an associated utility value. The number of dimensions of the space is given by the number of issues n under negotiation, and the number of dimensions of each constraint must be lesser or equal than n. The utility yielded by a given potential solution (contract) in the utility space for an agent is the sum of the utility values of all the constraints that are satisfied by that contract. Figure 1 shows a very simple example of an agent's utility space for two issues and three constraints: a unary constraint C1 and two binary constraints C2andC3. The utility values associated to the constraints are also shown in the figure. In this example, contract x would yield a utility value for the agent u(x)=15, since it satisfies both C1andC2, while contract y would yield a utility value u(y)=5, because it only satisfies C1.
More formally, we can define the issues under negotiation as a finite set of variables x = {x i |i = 1, ..., n}, and a contract (or a possible solution to the negotiation problem) as a vector s = {x s i |i = 1, ..., n} defined by the issues' values. Issues take values from the domain of Agent utility space is defined as a set of constraints C = {c k |k = 1, ..., l}.E a c hc o n s t r a i n t is given by a set of intervals which define the region where a contract must be contained to satisfy the constraint. In this way a constraint c is defined as ] defines the minimum and maximum values for each issue to satisfy the constraint. Each constraint c k has an associated utility value u(c k ).
A contract s satisfies a constraint c if and only if x s i ∈ I c i ∀i. For notation simplicity, we denote this as s ∈ x(c k ), meaning that s is in the set of contracts that satisfy c k . An agent's utility for a contract s is defined as u(s)=∑ c k ∈C|s∈x(c k ) u(c k ), that is, the sum of the utility values of all constraints satisfied by s. This kind of utility functions produces nonlinear utility spaces, with high points where many constraints are satisfied, and lower regions where few or no constraints are satisfied (Ito et al., 2008). For the purpose of this work, we make the following additional assumptions, relaxing some of the common limitations of previous approaches, which allows us to tackle more realistic scenarios: • We assume that the number of issues and the domains of the issues are such that they make exhaustive search within the utility space of the agents intractable. • We assume that the utility spaces of an agent is highly uncorrelated, and so no ap r i o r i assumptions may be made about where high utility contracts may be located. Therefore, agents may need to resort to local nonlinear optimization techniques to identify such high-utility contracts. • We assume knowledge about other agent's preferences not to be common (i.e. agents do not know their opponent preference structures, neither they can compute opponent's utility for a given contract). • We assume that the negotiation setting is competitive, and that agents may be unwilling to reveal too much information about their preferences to the other negotiating agents.
The proposed negotiation protocol and decision mechanisms, which are described in the next sections, are specifically designed to address this negotiation setting.

Expressive, iterative interaction protocol
In highly uncorrelated utility spaces, one of the main problems of basic auction-based negotiation is that it usually relies on a one-shot protocol: agents send their bids to the mediator, the mediator searches for solutions and the negotiation ends. If a solution has been found, the negotiation is successful. Otherwise, the only possibility is to repeat the process until it is successful. In scenarios where high-utility regions are "wide" this is hardly a problem, since agent bids usually cover a wide range of the solution space, and the probability for the mediator to find a successful deal is high. In highly uncorrelated utility spaces, however, high utility regions agents send as bids tend to be narrower, and thus it is very likely that a single iteration of the protocol achieves no solution. In these cases, it would be desirable that agents had a way to "learn" from previous interactions to issue bids which are more likely to result in a deal. For this to be possible, two mechanisms are needed: a mechanism for the mediator to provide feedback to the agents, and a mechanism which allows the agents to use this feedback in the generation of new bids.
Our proposal for an iterative, expressive protocol may be briefly described as consisting of the following steps: 1. Bidding:E a c ha g e n tA j generates a set B j with n j b bids. Each bid represents a region in the solution space which, in general, contains only contracts the agent would accept as solutions to the negotiation problem. 2. Deal Identification: The mediator employs a search method to find overlaps between agents' bids. If a solution is found, the protocol ends. 3. Feedback: If no satisfying solution is found, the mediator creates feedback statements ρ j for each agent and distribute them. Steps 2 to 4 are repeated until a solution is found or a given limit is reached (be it an amount of time or a number of iterations).
For the formal description of the interaction protocol, we have employed an specification framework based on formal dialogue games (Amgoud et al., 2000;Hitchcock et al., 2001;Lopez-Carmona, 2006;McBurney & Parsons, 2002), where the interaction protocol is modeled by defining three key elements. A set of locutions establishes which communications are permitted between the agents. A set of decision mechanisms allows agents to advance through the different stages of the protocol and to invoke the different locutions as needed. Finally, operational semantics indicates how the dialogue game evolves depending on the outputs of the different mechanisms and the locutions exchanged by the agents. The detailed specification of the protocol is beyond the scope of this discussion, and can be found in (Marsa-Maestre, 2009).
For the purpose of this chapter, to have a clear understanding of the role of the different elements within the protocol and the relationships between them will suffice. The dialogue may be seen as a set of states, where these states are defined by the different mechanisms, and where transition rules govern the transitions within the states when a certain locution is passed between agents or a certain output is passed between different mechanisms of the same agent. Figure 2 shows a state diagram which represents the link between transition rules, decision mechanisms, and locutions. The states represent the decision mechanisms, and the transitions are specified by the outputs of the mechanisms firing them and, in case of transitions involving different agents, by the exchanged locutions.
The protocol considers the existence of nine decision mechanisms: four for the negotiating agents (labeled Ax in the diagram) and five for the mediator (labeled Mx). Mechanisms M1, A1, M2,a n dM5 focus on the opening and closing of the negotiation dialogues, and are not of much interest for this specific work. Therefore, in the rest of the paper we will assume simple implementations for these mechanisms. On the other hand, mechanisms A2, A3, M3, and M4 play key roles in the negotiation stage of the protocol, and will be further discussed in the following section. Before addressing in detail the mechanisms directly related to the negotiation stage, we will briefly review the different mechanisms considered by the protocol: M1: Start Negotiation. Determines the initial feasibility of the negotiation. In the following, we will assume that all negotiations proposed by the mediator are feasible, and thus this mechanism will always yield as output feasible_negotiation. A1: Assess Negotiation. Decides whether the negotiating agent participates or not in the proposed negotiation. We will assume that not participating in the negotiation has zero utility for the agent, and so it will always participate. Thus, the output of this mechanism will always be enter_dialogue. M2: Confirm Dialogue. Confirms the feasibility of the negotiation once the responses from all agents have been received. Since the negotiation is defined as feasible in the output of the mechanism M1: Start Negotiation and all agents are willing to participate in the dialog, the output of this message will always be feasible_negotiation. A2: Generate Bids. Generates a set of bids to send to the mediator. This mechanism, along with mechanism A3: Assess Feedback, constitutes the core of the agent negotiation strategy. Therefore, in the following sections we will address different strategies for bid generation. M3: Identify Deals. Searches for solutions to the negotiation problem once the bids from the different agents have been received. The deal identification strategy at the mediator greatly influences important properties of the model such as social welfare maximization, deal probability and performance. M4: Generate feedback. Builds feedback statements sent to the negotiating agents to try to influence the generation of new bids at each protocol iteration. This mechanisms influences the ability of the model to refine the search for solutions in successive iterations. A3: Assess Feedback. This mechanism complements the previous one in the task to provide the model with the possibility for iterative improvement. It is used by negotiating agents to generate a new set of bids at each protocol iteration, taking into account the feedback received from the mediator. A4: Assess Solution. Decides whether the negotiating agent accept or rejects a solution found by the mediator. We will assume that negotiating agents have a reservation value which remains unchanged during the negotiation. A negotiating agent will accept any solution proposed by the mediator which is above its reservation value. M5: Confirm Solution. Confirms the success or failure of a negotiation once the responses from the agents have been received. We will assume that a negotiation is successful if all negotiating agents accept the solution proposed by the mediator.
For the negotiation model to be complete, it is necessary to define the negotiation mechanisms which will implement the agent strategies. Since this chapter is concerned mainly with mediated approaches and their use of heuristic methods and alternative social welfare notions, in the rest of this section we will focus in the mechanisms used to identify deals within the mediator (M3). Mechanisms used for bidding can be found in .

A probabilistic deal identification mechanism
Scalability is identified as one of the main drawbacks in a bidding based negotiation protocol (Ito et al., 2007). Once agents have placed their bids, the mediator performs an exhaustive search for overlaps between the bids using a breadth-first algorithm with branch cutting. In a worst case scenario, this means searching through a total of n n a b bid combinations, where n b is the number of bids per agent, and n a is the number of negotiating agents. In the experiments, the authors limit the number of combinations to 6, 400, 000. This means that, for 4 negotiating agents, the maximum number of bids per agent is na √ 6400000 = 50. This limit becomes harder as the number of agents increases. For example, for 10 agents, the limit is 4 bids per agent, which drastically reduces the probability of reaching a deal. This is specially true for highly uncorrelated utility spaces, where bids are narrower.
To address this scalability limitation, we propose to perform a probabilistic search in the mediator instead of an exhaustive search. This means that the mediator will try a certain number n bc of randomly chosen bid combinations, where n bc < n n a b .I nt h i sw a y ,n bc acts as a performance parameter in the mediator, which limits the computational cost of the deal identification phase. Of course, restricting the search for solutions to a limited number of combinations may cause the mediator to miss good deals. Taking this into account, the random selection of combinations should be biased to maximize the probability of finding a good deal. Intuitively, in such complex scenarios, a mediator with no knowledge of the agents' preferences could try to achieve that by balancing the utility of the agents' bids (to maximize agent's social welfare) and the volume of those bids (to maximize the probability of a successful negotiation).
More formally, we may find mathematic expressions for the deal probability and the expected utility in a negotiation using the auction-based protocol. The deduction of these expressions can be found in . For the purpose of this section, the final expressions will suffice. In particular, deal probability for a single run of the auction-based negotiation protocol is given by where n a is the number of negotiating agents, n is the number of issues, |D| is the domain size for the issues (assuming all issues have the same domain size), and n k bp is the number of bidden contracts for agent k, that is, an indication of the portion of the solution space which is covered by agent k bids. This is given by n k bp = ∑ n k b l=1 v k l ,w h e r en k b is the number of bids issued by agent k and v k l is the volume of each l-th bid. In a similar way, we can see that the expected utility for agent k is given by where u k l is the utility for the l-th bid of agent k. According to this expression, to maximize expected social welfare the mediator should try to maximize ∑ n k b l=1 u k l · v k l , balancing in this way bid utility and bid volume.
We formally represent this through the following definitions:

Definition 1. Volume of a bid. The volume of a given region b within the solution space D is defined as the cardinality of the set of contracts contained within the region defined by the bid. v(b)=|r| ,withb⊂ D
Definition 2. Quality factor. The quality factor of a given region b within the solution space D (be it a constraint or a bid) is defined as where u(b) and v(b) are, respectively, the utility and volume of the bid b.
Intuitively, it can be seen that the quality factor defined above allows an agent to balance bid utility (to maximize its own benefit) and bid volume (to maximize deal probability). Taking this into account, the parameter used to bias the random selection in the mechanism is Q 0 , so that higher-Q 0 bids have more probability of being selected for bid combinations at the mediator.
The mechanism is formally shown in Algorithm 1. We can see that the number of analyzed bid combinations is limited to n bc (1), and that the function combine_bids (...) selects the bid combinations to analyze (2). Limiting bid combinations at the mediator allows us to remove the limit on the bids issued by the agents, which increases the probability of finding good potential deals. However, it may be convenient to keep a less strict limit in the number of bids sent to the mediator, in order to keep communication load of the negotiation process within reasonable bounds.

Input:
A: set of negotiating agents; n a = |A|: number of negotiating agents B: set of bids issued by every agent; U: declared utilities for every agent's bids Q 0 : declared quality factors for every agent's bids; sw: social welfare function n bc : maximum number of bid combinations at the mediator Output: s f : final potential deal n = 0; S = ∅; 1 while n < n bc do 2 s = combine_bids (A, n a , B, U, Q 0 ) ; if s = ∅ then S = S ∪ s; n = n + 1; end s f = arg {max s∈S sw (s, U)}; The core of this mechanism from an strategic point of view is the probabilistic selection function combine_bids (A, n a B, U, Q 0 ), which generates a bid combination selecting a bid from each agent. This function is the one which ultimately determines which bid combinations are tested at the mediator. Different algorithms may be used to generate these combinations. Among the multiple possibilities, we have considered the following ones: • Uniform combination selection: The simplest approach is to make a totally random selection of bids for each combination. Intuitively, the main drawback of this approach is that, as discussed above, reducing the search space arbitrarily may lead us to discard high social welfare solutions. • Tournament selection of bid combinations: It seems reasonable to bias combination selection by means of the quality factor Q, that is, making the selection favor bids with high quality factor. A fairly straightforward way to do this is by means of tournament selection.F o r each bid combination, a subset of n t bids is randomly drawn from each agent, and the bid maximizing quality factor is selected from each subset to form the combination. • Roulette selection of bid combinations: In this case, to select a bid for an agent j,asetoflabels L j is created, where each label l j i corresponding to bid b j i is repeated a number of times proportional to the bid quality factor. Once the label set has been generated, a random is generated for each agent j, and the bid corresponding to label k j -thinsetL j is selected. In this way, the probability for each bid to be selected is proportional to its quality factor Q 0 . • Ordinal probabilistic selection: For each agent j,t h eb i ds e tB j is sorted in quality factor Q 0 descending order. Once the set has been sorted, a random number k j ∈ 1, n j B is drawn from a probability distribution favoring lower numbers, and the k j -th bid is selected. Some probability distribution functions which may be used are the negative binomial distribution, the geometric distribution (a particular case of the previous one), or the positive normal distribution.
As we can easily see, by using this strategy for deal identification, the mediator is favoring those bids combinations which maximize the sum of quality factors, rather than maximizing the sum of utilities. In this sense, the mediator is using an alternative form of social welfare, which not only cares about individual agent utility, but also about deal probability. Experiments performed in  show that using this Q-based social welfare notion allows for better performance and scalability in complex utility spaces. However, the protocol and deal identification mechanism are prone to some strategic stability problems, which may make individually rational agents agents to try to bid narrow, high utility bids to maximize their expected utility in case of agreement, but severely impacting deal probability. These stability issues can also be lowered by means of alternative social welfare, as we will see in the following section.

Addressing stability issues through the use of alternative social welfare notions
In , once a set of viable solutions has been found, the mediator chooses as the solution the one which maximizes social welfare. However, as discussed in , when the mediator uses the classic social welfare option an agent's attitude greatly influences the final utility value for this agent if an agreement is reached. Once all valid intersections have been found, the final outcome is selected using a function which depends on the utility values the outcome gives to the agents. Selfish, risk-willing or highly competitive agents, which give more importance to utility against volume in the bid generation process, will have, on average, higher utility bids, and thus their expected utility conditioned to the existence of a deal will be higher. Taking this into account, the preferred strategy of an agent may be to take a selfish attitude, as we inferred in the previous section. The problem is that, in complex utility spaces, having all agents taking such attitudes could lead to very narrow offers at the mediator, which would make deal probability (given by Equation 1) decrease drastically. This may lead the protocol to negotiation failures, with zero social welfare, thus resulting in situations of infinite Expected Price of Anarchy (Papadimitriou, 2001) , turning the negotiation model unstable.
To improve the strategic stability of the negotiation, the mechanisms should be modified to incentivize the adoption of socially optimal strategies. A feasible way to do that is to modify the social welfare notion at the mediator to reward in the selection of the final solution to those agents which have made wider bids. This can be done by using a generalized or asymmetrical version of the Nash product, similar to the ones used in (Kalai, 1977) to model agents' power of commitment. In particular, we propose a modification of the Nash product which we have called weighted product by average volume: Definition 3. Weighted product by average volume The weighted product by average volume of a solution to a negotiation problem among n a agents is the product of the utilities the solution gives to every agent i, weighting each utility u i (s) by an adjustment factor equal to the ratio between the average volume of the bids issued by the agentv i and the maximum average volume of the bids of one of the agents:

where u i (s) is the utility of the solution s for agent i, andv i is the average volume of the bids issued by agent i.
In this way, the utility for those agents who have issued widest bids (which, on average, will be the ones using more socially oriented strategies) will be given more weight in the selection of the final solution than those of the more selfish agents. An interesting effect of this metric is that a rational agent could issue some high volume, low utility bids to try to compensate for its high-utility, low volume bids. To counter this effect, we propose to consider bid utility and bid volume jointly, using a product weighted by average quality factor: Definition 4. Weighted product by average quality factor The weighted product by average quality factor of a solution to a negotiation problem among n a agents is the product of the utilities that solution gives to every agent i, weighting each utility u i (s) by an adjustment factor equal to the ratio between the average quality factor of the bids issued by the agentQ i 0 and the maximum average quality factor of the bids of one of the agents: whereQ 0 i is the average quality factor of the bids issued by agent i.
In , experiments were conducted for the different social welfare notions, comparing the utility obtained by a selfish agent, using its individually optimal strategy, against the utility obtained by the agent when using the corresponding socially optimal strategy, assuming the rest of the agents are also using that socially optimal strategy. Experiments were made for utility spaces with different correlation lengths. Furthermore, since the model is designed for multi agent negotiations, experiments were performed for different number of selfish agents, thus studying the effect of possible coalitions or coincidences. Results showed that the mechanism based on average volume provided not enough improvement in stability. For all cases median utility results were higher for selfish agents, thus maintaining the incentive for agents to deviate from the socially optimal strategy. The mechanism based on average quality factor, however, significantly mitigated the gain for selfish agents, removing the incentive to choose the previously individually optimal strategy.
Another threat to mechanism stability is strategic revelation of information. In incomplete information scenarios (Jonker et al., 2007), since the agents' beliefs about the preferences of a given agent may influence the decision mechanisms they use, an agent may use as a strategy to lie about its own preferences in order to manipulate the decision mechanisms of the rest of the agents to its own benefit. This raises an additional concern to mechanism design (Weiss, 1999). It would be desirable to design protocols which are not prone to be manipulated through insincere revelation of information. Incentive-compatibility is defined as the property of a negotiation mechanism which makes telling the truth the best strategy for any agent, assuming the rest of the agents also tell the truth.
In the negotiation model we are dealing with, information revealed to the mediator is the set of agents' bids. These bids represent regions within the solution space. Each offer has an associated utility value, a volume, and an associated quality factor value. Since bid volume is directly related to the region represented by the bid, it does not seem feasible to fake it, since it can be easily checked by the mediator. Quality factor may be faked, but since the mediator is very likely to recompute it, this strategy is also harmless. Finally, agents may fake bid utility. Insincere information revelation about bid utility may generally occur in two ways: exaggerating upward or downward the utility values of all bids, or exaggerating the utility values of some bids with respect to the others. Exaggerating all bids is not profitable with the proposed deal identification mechanisms, since bid selection at the mediator is performed independently for each agent. This means that the bids from different agents do not compete among each other to be selected as part of a solution. In contrast, the different bids of a single agent compete among themselves. Taking this into account, an agent could try to exaggerate the utility value of its preferred bids, thus trying to increase the probability of the mediator choosing those preferred bids to form deals. As far as social welfare is concerned, this is a problem if the set of exaggerated bids is small with respect to the total set of bids, since that would reduce the number of effective bids considered by the mediator, thus reducing deal probability.
Therefore, we seek for mechanisms which counter this effect, incentivizing sincere revelation of information. A possibility to achieve this is to normalize the utility values assigned by the agents to their bids, thus lowering the absolute differences in utility. We propose three different possibilities regarding utility normalization: • Normalization to maximum utility : obtained by dividing each agent's bid utility by the maximum utility value issued by that agent: Using this normalization mechanism we can avoid the manipulation of the final deal by exaggerating upwards the utility values of the preferred offers. It does not prevent, however, downward exaggerations, that is, to assign an extremely low value to the bids which are less profitable for the agent. • Bounded maximum-minimum normalization: Attempts to prevent the manipulation of the negotiation model through upwards or backwards exaggerations. It is given by the expression where u max = max b j ∈B u b j , u min = min b j ∈B u b j and u ′ min and u ′ max are parameters chosen by the mediator. In this way, a utility mapping from the interval [u min , u max ] to the interval u ′ min , u ′ max is performed for all bids, putting an upper bound to the ratio between the utilities of an agent's bids.
• Ordinal normalization: obtained by ordering the different bids of an agent according to their utility or quality factor, and mapping this order to a monotonically increasing succession of utility values, regardless of the original utility values. For instance, if B is the set of bids for an agent, in ascendent order of utility, and taking the arithmetic succession s = {1, 2, . . . , n B } as the mapping function, the normalized bid utility values would be of the form To evaluate the effect of the described mechanisms over the incentive-compatibility of the model, in  we also conducted experiments comparing the utility obtained by an insincere agent with the utility obtained being sincere, assuming the rest of the agents are sincere. The behavior of the insincere agent was modeled by exaggerating the utility of a portion of the agent's highest utility bids. We considered different degrees of exaggeration for the insincere agent: the insincere agent exaggerated 75%, 50%, 25% and 12.5% of its bids. Results showed that all proposed normalization techniques reduced the incentive for the insincere agent to exaggerate, but only bounded maximum-minimum normalization made the expected gain for the insincere agent negligible, thus effectively removing the incentive to exaggerate, improving incentive-compatibility of the model.

Using social welfare to incentivize different consensus policies
So far we have seen how social welfare notions may be used to incentivize different kind of behaviors from the negotiating agents. However, social welfare notions can also be used to directly influence the negotiation outcome, looking for specific kinds of agreements. In , a consensus policy mediated framework (CPMF) is defined for negotiations with complex utility spaces. The negotiation framework is based on the use of a mediator, which iterates through a series of contracts trying to optimize social welfare throughout the use of Mesh-adaptive Generalized Pattern Search (Lewis et al., 2000) and an iterative, expressive protocol which gathers the different negotiating agents valuations for the contracts in the mesh. This framework is specially interesting for the scope of this chapter, because the framework explicitly allows the mediator to pursue optimization objectives different than classic social welfare, thus allowing for the enforcement of different consensus policies.
The most widespread consensus policy found in the automated negotiation literature is social welfare maximization, which suggests desire to satisfy all the agents. However, the requirement to satisfy all the agents may not be suitable for multi-agent preference aggregation, or simply the system may need to implement more sophisticated forms of aggregation.
We propose to use other mediation rules to improve the negotiation processes where either a quorum is not necessary or simply such quorum is not possible. For example, a solution may be acceptable if most of the agents support it. To incorporate these notions into our negotiation framework we will use a more general class of aggregation rules. The idea is to use a quantifier guided aggregation, which allows a natural language expression of the quantity of agents that need to agree on an acceptable solution. As we shall see the Ordered Weighted Averaging (OWA) operator (Yager, 1996) will provide a tool to model this kind of softer mediation rule.

OWA operators
An aggregation operator M : S n → G, (S, G ∈ [0, 1]) is called an OWA operator of dimension n if it has an associated weighting vector W =[w 1 w 2 ...w n ] such that w t ∈ [0, 1] and ∑ n t=1 w t = 1a n dw h e r eM(S 1 ,...,S n )=∑ n t=1 w t b t where b t is the tth largest element of the aggregates {S 1 ,...,S n }.
Note that in the definition of OWA we have used the notation M to identify the aggregation operator with the mediation rule, S n to make reference to the preferences of the agents, and G to define the group preference. In the OWA aggregation the weights are not directly associated with a particular argument but with the ordered position of the arguments. If ind is an index function such that ind(t) is the index of the tth largest argument, then we can express M(S 1 ,...,S n )=∑ n t=1 w t S ind (t) . It can be shown that OWA aggregation has the following properties: Under these conditions the OWA operator is a mean operator. The form of the aggregation is dependent upon the associated weighting vector. We have a number of special cases of weighting vector are worth noting. The vector W * defined such that w 1 = 1a n dw t = 0for all t = 1 gives us the aggregation Max i [S i ]. Thus, it provides the largest possible aggregation. The vector W * defined such that w n = 1a n dw t = 0f o ra l lt = 1 gives the aggregation Min i [S i ]. The weighting vector W ave defined such that w t = 1/n g i v e su st h ea v e r a g e 1 n ∑ n i=1 S i . Finally, an interesting family of OWA operators are the E-Z OWA operators. There are two families. In the first family we have w t = 1/q for t = 1toq,andw t = 0fort = q + 1to n. Here we are taking the average of the q largest arguments. The other family defines w t = 0 for t = 1t oq,a n dw t = 1 n−q for t = q + 1t on. We can see that this operator can provide a softening of the original min and max mediation rules by modifying q.

Quantifier guided aggregation
In the preceding, we have seen how the OWA operators can be used to compute the group preference for different alternatives. However, our aim is to define consesus policies in the form of a linguistic agenda for our mediation mechanisms. For example, the mediator should make decisions regarding the exploration of the negotiation space, i.e. expansion and contraction of the mesh, following mediation rules like Most agents must be satisfied by the contract, at least α agents must be satisfied by the contract, many agents must be satisfied, . . . The above statements are examples of quantifier guided aggregations. (Zadeh, 1983) suggested a formal representation of these linguistic quantifiers using fuzzy sets. He suggested that any relative linguistic quantifier can be expressed as a fuzzy subset Q of the unit interval I =[ 0, 1]. In this representation for any proportion y ∈ I, Q(y) indicates the degree to which y satisfies the concept expressed by the term Q. In most applications of the quantifier guided aggregation we use a special case class of these liguistic quantifiers, called Regular Increasing Monotone (RIM) quantifiers. These types of quantifiers have the property that as more agents are satisfied our overall satisfaction can't decrease. Formally, these quantifiers are characterized in the following way: 1) Examples of this kind of quantifier are all, most, many, at least α. Two examples of RIM quantifiers are all which is represented by Q * where Q * (1)=1andQ * (x)=0forallx = 1, and any which is defined as Q * (0)=0andQ * (x)=1forallx = 0.
The question now is how to obtain the OWA operator to satisfy a quantifier guided aggregation. Again assume we have a collection of n agents. These agents have their preferences represented as fuzzy subsets over the set of alternatives in the mesh {S 1 (x +o (k)),...,S n (x +o (k))}. Under the quantifier guided mediation approach a group mediation protocol is expressed in terms of a linguistic quantifier Q indicating the proportion of agents whose agreement if necessary for a solution to be acceptable. The basic form of the mediation rule in this approach is that Q agents must be satisfied by the contract, where Q is aquantifier .
The formal procedure used to implement this mediation rule is described in the following. The quantifier Q is used to generate an OWA weighting vector W of dimension n. This weighting vector is then used in an OWA aggregation to determine the group support for the contract. For each contract in the mesh the argument of this OWA aggregation is the degree of support for that contract by each of the agents, S i (x e j (k)), i = 1,...,n. Thus, the process used in the quantifier guided aggregation is as follows: 1. Use Q to generate a set of OWA weights, w 1 ,...,w n . 2. For each contract x e j (k) in x +o (k) calculate the overall group support G(x e j (k)) = M(S 1 (x e j (k)),...,S n (x e j (k))).
The procedure used for generating the weights from the quantifier is to divide the unit interval into n equally spaced intervals and then to compute the length of the mapped intervals using Q Because of the nondecreasing nature of Q it follows that w t ≥ 0. Furthermore from the regularity of Q, Q(1)=1a n dQ(0)=0, it follows that ∑ t w t = 1. Thus we can see that the weights generated are an acceptable class of OWA weights.
In Figure 3 we show an example of a RIM linguistic quantifier and illustrate the process of determining the weights from the quantifier. We see that the weights depend on the number of agents as well as the form of Q. In Figure 4 we show the functional form for the quantifiers all, any, Q * , Q * , at least α percent, linear quantifier, piecewise Q Z β and piecewise Q Z α . The quantifiers all, any and at least α describe the consensus policy using a natural language verbal description. However, more generally any function Q : [0, 1] → [0, 1] such that Q(x) ≥ Q(y) for x ≥ y, Q(1)=1andQ(0)=0 can be seen to be an appropriate form for generating mediation rules or consensus policies. Thus there are two techniques to generating these quantifier based mediation rules. One possibility is to start with a linguistic expression and then obtain Q. The second approach is to allow the mediation rule to be directly expressed in terms of a function Q. One important characteristic of this second method is that we can easily introduce into our mediation a number of formal properties that are not very easily expressed using a verbal description of the quantifier. The linear quantifier Q(y)=y for instance generates w t = 1/n, and thus, all the agents get the same weight. The Q Z β quantifier it is required that at least β agents are satisfied to initiate a Q linear improvement. Q Z α initiates the Q linear improvement with the first satisfied agent, and once there are α agents satisfied there is no improvement in Q if more agents are satisfied.
One feature which distinguishes the different types of mediation rules is the power of an individual agent to eliminate an alternative. For example, in the case of all this power is complete. In order to capture this idea the Value Of Individual Disapproval (VOID) measures this power. For the all, any, at least α and linear quantifiers the VOID measures are respectively 1, 0, α and 0.5. For the Q Z β quantifier VOID(Q Z β )= 1 2 + β 2 ) and therefore VOID(Q Z β ) ∈ [0.5, 1].T h eQ Z α quantifier gets VOID(Q Z α )= α 2 and VOID(Q Z α ) ∈ [0, 0.5]. Another family of quantifiers are those defined by Q p (y)=y p for p > 0. In this case VOID(Q p )=1 − 1 0 r p dr = p p+1 . For this quantifier we can easily obtain the OWA weights with For Q p we see that as p increases we get closer to the min and that as p gets closer to zero we get the max.

Impact of consensus policies on automated negotiations
In , we performed experiments to test the performance of the protocol under a proof of concept scenario and a complex negotiation scenario for 5 different consensus policies defined by the corresponding VOID degrees: 0, 0.25, 0.5, 0.75 and 0.95, using the quantifier Q p (y)=y p . We also defined a contrast experiment where the consensus policy based mediation process is deactivated, such that the group preference evaluation is limited to compute the sum of agents' valuations for a given contract (i.e. the winner contract is that with the highest sum of valuations), which corresponds to classic social welfare. Experiments showed that social welfare maximization tends to achieve poorer agreements in complex utility spaces, while consensus policies which try to satisfy only a subset of the agents achieve higher satisfaction degrees for the agents. Using different consensus policies may allow, for instance, to perform simultaneous partial agreements among subsets of the negotiating agents, which may be of key importance in some scenarios, like group formation negotiations.

Concluding remarks
In this chapter we have studied how different notions of social welfare may be used to influence the outcome of mediated negotiations. First, we have presented a model for automated negotiation with complex, constraint based utility spaces, where using social welfare notions which take into account both bid utility and deal probability provide an improvement of the performance and scalability of the negotiation protocol as the complexity of the utility space increases. Results show that there is a significant advantage in using the quality factor Q 0 and the proposed mechanisms in highly uncorrelated utility spaces, both in terms of optimality and failure rate. We have also seen how social welfare notions can address strategic issues within our model, allowing to address important problems such as stability and incentive compatibility.
The model definition has been divided into three parts: the agent preference model, the interaction protocol and the decision mechanisms. To describe the agent preference model we have chosen a representation based on weighted constraints, since this allows to easily generate utility spaces of adjustable complexity, which is an advantage for the evaluation of the proposed mechanisms. Regarding the interaction protocol, an auction-based, expressive, iterative approach has been chosen. The use of the mediator allows to reduce information revelation between negotiating agents, and also facilitates to separate the individual agent strategies (which for rational agents tend to be maximize their own profit) from the social strategies. The interaction protocol is closely related to the one-shot, auction-based protocol described in (Ito et al., 2007), with the difference that our model considers the possibility to perform multiple iterations and provides the mediator with expressive capabilities to send feedback to the agents at each iteration, in order to refine their bids. There are other works in the literature suggesting the use of expressive negotiation protocols in multi-agent negotiations. (Li et al., 2009b;Marsa-Maestre et al., 2008) use gradient information to bias the search for solutions in linear unmediated negotiation, and (Lopez-Carmona et al., 2007) uses relax requirements in bilateral buyer-seller negotiations.
Finally, we have explored the hypothesis that the consensus type by which an agreement meets in some specific manner the concerns of all the negotiators should be considered in the construction of multiparty negotiation protocols. We argue that there exist situations where an unanimous agreement is not possible or simply the rules imposed by the system may not seek such unanimous agreement. Thus, we have used a particular aggregation of the agent preferences by the mediator, which allows to define different social welfare notions (which we call consensus policies) based on OWA and quantifier guided aggregation. Results show that CPMF efficiently manages negotiations following predefined consensus policies and solves situations where unanimous agreements are not viable.
Though the conducted experiments have yielded satisfactory results, there is still plenty of research to be done in this area. We believe that the results presented opens the door to a new set of negotiation policies where social welfare notions, be it as behavior adherence metrics or as consensus policies, will play an important role. We are also interested in dynamic adjustments for the social welfare notions, so that at different steps in the negotiation the consensus policies may be adapted to the specific negotiation dynamics. Finally, the challenge of enforcing alternative social welfare notions without the aid of a mediator still remains open.

Acknowledgments
This work has been supported by the Spanish Ministry of Education and Science grants: TIN2008-06739-C04-04, T2C2 research project, and the mobility grant JC2010-003.