Autonomic Solutions for Virtualised ICT Systems more

by Peter R Lewis

Unpublished. Doctoral thesis proposal.

Autonomic Solutions for Virtualised ICT Systems Peter R. Lewis Thesis Group 3 Report: Thesis Proposal Supervised by Professor Xin Yao and Dr Paul Marrow Thesis Group Members: Dr John Bullinaria, Dr Jonathan Rowe August 2007 Abstract The environment within which information and computation systems operate is changing. High bandwidth communication channels are everywhere and telecommunications infrastructure is continuing to be deregulated around the world. Both newcomers and more established companies are increasingly ﬁnding themselves operating in scenarios where they do not have full control or indeed full knowledge of key resources such as network infrastructure, data centres and computational capacity. These are resources which nonetheless remain vital to their continued successful operation and development. In this context, paradigms are being created for the development and deployment of massively distributed computational and business systems, spanning many locations, organisations and platforms. Service-oriented computing is one such paradigm. In order for any organisation to work successfully within such a computational ecosystem, there is a need to ﬁnd novel ways to manage, control and understand such highly complex distributed systems. In any scenario where ﬁnite resources are required for the completion of activities, a method must be found for their allocation. In the human world, building on the simple idea of voluntary bilateral exchange, many cultures have developed the use of free markets as their primary method of resource allocation. Markets are now also common in the electronic world, and are used not only to aid humans in their transactions, but also in purely electronic scenarios, where computational, informational and communication resources may be traded. This proposal lays out directions for research into the equilibria and dynamics of computational markets. With a focus on evolutionary analysis and design, markets contatining both absolute and geographical trade barriers are discussed, and methodologies described for their analysis. This analysis will be used to inform the development of high performing evolutionary selling strategies, in order that eﬃcient allocations of resource may be achieved in dynamic scenarios. A comprehensive introduction, background discussion and review of related literature are presented. Research hypotheses, objectives, success criteria and a timetable are then described. Contents 1 Introduction 2 Background and Analysis of Previous Work 2.1 Motivation . . . . . . . . . . . . . . . . . . . 2.2 Service-Oriented Computing . . . . . . . . . . 2.2.1 A Market For Services . . . . . . . . . 2.3 Free Markets . . . . . . . . . . . . . . . . . . 2.3.1 A Simple Example . . . . . . . . . . . 2.3.2 Measuring Utility And Optimality . . 2.3.3 Auctions . . . . . . . . . . . . . . . . . 2.3.4 An Economy Of Agents . . . . . . . . 2.3.5 Decentralised, Not Just Distributed . 2.3.6 Marketing Strategies . . . . . . . . . . 2.3.7 Trade Barriers . . . . . . . . . . . . . 2.4 An Evolutionary Approach . . . . . . . . . . 2.4.1 Understanding The Market . . . . . . 2.4.2 Evolutionary Marketing Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 4 4 5 6 7 7 8 9 10 12 12 14 14 15 15 18 18 18 19 19 21 22 23 24 25 25 25 27 28 3 Statement Of Work To Be Done 3.1 The Problem Area . . . . . . . . . . . . . . . . 3.2 The Work Ahead . . . . . . . . . . . . . . . . . 3.3 Investigation . . . . . . . . . . . . . . . . . . . 3.3.1 Problem Speciﬁcation . . . . . . . . . . 3.3.2 An Evolutionary Investigative Approach 3.3.3 Partially Connected Markets . . . . . . 3.3.4 Geographically Separated Markets . . . 3.3.5 Outcomes . . . . . . . . . . . . . . . . . 3.4 Development . . . . . . . . . . . . . . . . . . . 3.4.1 Initial Experiments . . . . . . . . . . . . 3.4.2 Going Forward . . . . . . . . . . . . . . 3.5 Evaluation Of The Work . . . . . . . . . . . . . 3.6 Timetable . . . . . . . . . . . . . . . . . . . . . 1 Chapter 1 Introduction In any scenario where ﬁnite resources are required for the completion of activities, a method must be found for their allocation. In the human world, building on the simple idea of voluntary bilateral exchange, many cultures have developed the use of free markets as their primary method of resource allocation. Markets are now also common in the electronic world [33], and are used not only to aid humans in their transactions, but also in purely electronic scenarios, where computational, informational and communication resources may be traded. One emerging area of Computer Science which stands to beneﬁt from the development of purely electronic markets is service-oriented computing. In this rapidly developing and important ﬁeld, resources are componentised and exposed as services, available for use by other parties in carrying out their activities [58]. Key to the idea of service-oriented computing is the ability to dynamically compose applications from generic and reusable services. Comparable services may be oﬀered by many providers, enabling service users to select at run-time the provider that best suits their needs. This just-in-time approach in turn allows for service providers to dynamically adjust the properties of the services they oﬀer to achieve their own aims. With service providers competing for ﬁnite users and service users attempting to perform their activities to the best of their abilities, we have indeed an interesting and social resource allocation problem. Indeed, research in service-oriented computing has looked to social and marketbased systems to solve its resource allocation problems [58]. However, since there is a desire for the next generation of service-oriented architectures to be fully decentralised, without the fragility brought about by centralised registries and brokers [49], novel approaches are needed for the eﬃcient and appropriate allocation of resources in these architectures. Market-based control techniques are increasingly being used as approaches to distributed resource allocation, most examples making use of classic auction mechanisms [13]. Some work exists on the design of distributed auctions [25, 22, 41, 40, 30, 31, 32], though it has been questioned to what extent many market-based control approaches are decentralised [16]. Nevertheless, Kuwabara et al have demonstrated that eﬃcient allocations can be achieved under certain conditions in a fully decentralised system, without the overhead of an auction [42]. Drawing on evolu2 tionary optimisation and learning techniques, particularly the co-evolution of trading agents, this thesis will aim to contribute to the further development of this approach. Tools such as evolutionary game theory will aid the analysis and deeper understanding of its applicability, strengths and weaknesses. It is hoped that this work will contribute to the development of eﬃcient and well understood resource allocation and control mechanisms for fully decentralised service-oriented architectures. 3 Chapter 2 Background and Analysis of Previous Work 2.1 Motivation The environment within which information and computation systems operate is changing. High bandwidth communication channels are everywhere and telecommunications infrastructure is continuing to be deregulated around the world. Both newcomers and more established companies are increasingly ﬁnding themselves operating in scenarios where they do not have full control or indeed full knowledge of key resources such as network infrastructure, data centres and computational capacity. These are resources which nonetheless remain vital to their continued successful operation and development. Meanwhile, ever more distributed and accessible resources mean that smaller organisations and individuals are able to access resources previously unavailable to them, or indeed to provide specialist business services to a wider audience than previously conceived, at little extra cost. In this context, paradigms are being created for the development and deployment of massively distributed computational and business systems, spanning many locations, organisations and platforms. Service-oriented computing is one such paradigm. In order for any organisation to work successfully within such a computational ecosystem, there is a need to ﬁnd novel ways to manage, control and understand such highly complex distributed systems. Service-oriented computing research has to date looked to socially and economically inspired agent-based systems to manage the selection and composition of distributed components in a necessarily dynamic environment [58]. This exciting approach is by its very nature interdisciplinary. Whilst the application area is ﬁrmly rooted in ICT, the techniques involved draw heavily from microeconomics, game theory and evolutionary biology. 4 2.2 Service-Oriented Computing Service-oriented computing is a major new paradigm for the development and deployment of large information and computation systems. Motivated by organisations seeking to move away from traditional monolithic enterprise systems, individual processes and resources are instead isolated and expressed as components. Along with an architecture to support the dynamic composition of these components, this approach enables organisations to specialise and collaborate at many levels [58]. The components themselves are described at a suﬃciently abstract level to enable them to contain almost any type of business process or resource. Components might fall into one of three categories: • Computational resource (e.g. processor intensive algorithm, or raw CPU cycles) • Informational resource (e.g. remote storage, a database or access to privileged information) • Infrastructure resource (e.g. network bandwidth or a satellite link) Components may then be described and published, to enable them to be discovered and composed into an application. When a component is utilised in this way, through a supporting service-oriented architecture, the component is referred to as a service. There is a wealth of literature concerning the speciﬁcation and design of service description, publication and discovery. Huhns and Singh’s book [58] provides an excellent starting point, though these issues are not our primary concern here. One key concept in service-oriented computing is that of granularity. Huhns and Singh deﬁne four basic abstraction levels [36]: • Intra-enterprise, • Inter-enterprise, • Infrastructure, • Software component. At the intra-enterprise level, services are exposed only within the organisation’s own system. In this internal environment, the paradigm may be used to eﬃciently allocate the organisation’s resources to achieve its own goals. The inter-enterprise level is perhaps very similar, albeit at a coarser level of granularity. In this abstraction, we are interested in the services exposed externally by organisations, and the dynamic composition of them to form crossbusiness applications. Though technically similar to the intra-enterprise abstraction, here we are dealing with components beyond our control and outside of our ownership. This in turn gives rise to issues of trust (and perhaps reputation), cooperation and competition. However, it is also worth pointing out 5 that although the intra-enterprise abstraction may be seen to be a “safer” environment, the issues arising from the inter-enterprise abstraction may also apply when considering quality of service. The infrastructure abstraction level deals not with business processes and resources, but instead with the infrastructure resources which will be required to support and carry the communications used at both intra-enterprise and inter-enterprise levels. Aside from the diﬀerence in service-type, there is not a great deal of diﬀerence in principle between this abstraction and the former two. Finally, the software component abstraction provides us with the necessary building blocks to be able to construct cross-organisational distributed applications, ﬂexibly composed at run-time. By viewing services as software components, the aim is to be able to design and engineer such applications using familiar development methodologies. Furthermore, granularity permits each composed application to itself be described and published as a service, and used in the composition of larger applications. The approach is therefore, in theory, highly scalable. Due to this ﬂexibility and scalability, service-oriented architectures are becoming the prominent paradigm for distributed computing and e-commerce [65]. Indeed, some analysis claims that the global market in service-oriented computing will be $18.4 billion by 2012 [19]. Nevertheless, the paradigm is one very much still in development, with high proﬁle seminars producing comprehensive research roadmaps [49, 35]. 2.2.1 A Market For Services When composing an application in a service-oriented architecture, appropriate services may be made available by a wide range of service providers, from either inside or outside the organisation. Similarly, a variety of diﬀerent users may wish to access the services on oﬀer. Although each oﬀered service may be suitable, comparable services may not be identical. In selecting a service provider, a trade-oﬀ may exist between quality of service issues such as bandwidth, accuracy, reliability and latency. Furthermore, these issues may need to be balanced against cost. These factors may all change over time, between application compositions or indeed during an application’s lifetime. The interactions between multiple service providers and users may be viewed as a marketplace [48]. In this market providers must learn to oﬀer an appropriate quality service at the best cost. Given limited supply, users may compete for scarce service resources. Conversely, in a supply-heavy market, providers may adjust their services in order to compete for a few valuable customers. This should all take place transparently. Indeed, a well-performing marketplace should ensure the eﬃcient operation of the applications composed through it. This is a common theme in the market-based control of complex systems [13]. 6 2.3 Free Markets As mentioned earlier, free markets are based on the simple idea of voluntary (and usually) bilateral exchange. For a voluntary exchange to be valuable to either party, it is necessary ﬁrstly for each individual to have some notion of the value or utility he attributes to each good in question, and secondly for there to be two or more goods involved [60]. 2.3.1 A Simple Example In order to illustrate this, imagine Alice and Bob each have ten loaves of bread in their possession, which they are free to exchange with each other should they wish. Assuming that both value bread equally, and that they are both capitalists (i.e. they wish to maximise their own wealth), any exchange voluntarily entered into by both should not leave either party with less than the initial ten loaves. Indeed, there is nothing to be gained or lost by entering into an exchange, voluntary or otherwise. Furthermore, if a cost was to be associated with performing the exchange (such as a journey to visit the other), at least one would be worse oﬀ than before, and their combined wealth would also have reduced. Now suppose that for some reason Alice, who lives in an area where bread is hard to come by, values bread twice as much as Bob does. In this case, subjectively, the total value of the bread would be maximised by it all being in Alice’s possession. Clearly, however, Bob is unlikely to agree to this, since his own wealth would be reduced by this exchange. Therefore, given voluntary trade, no exchange would take place, yet the resources in the system would not be allocated in order to maximise their value. Let us introduce a second good, milk, and suppose that both Alice and Bob are initially endowed with ten jugs each, which they value equally. Now, we ﬁnd that a voluntary trade may take place. Bob is willing to let Alice have his ten loaves of bread in exchange for her ten jugs of milk. After completing the trade, Alice has twenty loaves of bread (which she values highly) and is better oﬀ than before the exchange, while Bob has twenty loaves of bread and is no worse oﬀ than before. Clearly, should Bob have valued milk more than Alice, then the trade would also have left him better oﬀ. In either case, allowing voluntary exchange allocates the available resources (bread and milk) in a manner which maximises the total value in the system. However, ﬁnding an optimal allocation is invariably more complex than this. The value an individual attributes to the goods available in a proposed trade (their utility), will most likely vary according to a number of factors. In our example, Bob may value milk more than bread, but this may only hold whilst he has enough bread to eat. Once he has traded away eight of his ten loaves for Alice’s milk, his preferences may shift, favouring holding onto his remaining two loaves of bread. Their relative utility to him have increased. It is clear that utility functions may be highly non-linear. 7 This kind of micro-economic system was investigated in simulation by Steiglitz et al [60]. Posessing two goods, food and gold, agents were able to trade to satisfy their own preferences. An agent’s utility function depended on its current inventory and the previous day’s price. Even with such a minimal system, they observed both periodic and “seemingly chaotic” ﬂuctuations in the price of food (in terms of gold). They further observed that these ﬂuctuations could be stabilised somewhat by the presence of agents who were capable of speculating on future prices, when determining what to trade. 2.3.2 Measuring Utility And Optimality Despite the ﬂuctuations in price observed by Steiglitz et al, traditional economic theory describes an economic system as being akin to a physical system, posessing an equilibrium point, towards which the system will drift, and at which the system is at rest [5]. Adam Smith further postulated that what he called an invisible hand would drive the market not only towards equilibrium, but that this would be the point at which the total utility of the system is maximised [59]. However, utility is highly subjective. Indeed, the concept of scalar measurement for utility (the util was originally proposed as a unit of measurement) was quickly discarded and it is now more commonly thought of as an expression of relative preference by an individual between two or more options [5]. In the real world, we may not have access to the knowledge of how individuals value diﬀerent goods at diﬀerent times and in diﬀerent circumstances (their utility functions). It is therefore often not possible to determine, from a central point of view, how to allocate resources optimally, or indeed when the task has been achieved successfully. Utility, it seems, is not a currency in itself, and is more helpfully thought of as being akin to relative happiness with ones lot when compared to that of another. Interestingly, the psychologist Daniel Nettle argues that increasing ones monetary income often has very little or no enhancing effect on ones happiness. He notes that in the United States, despite an average increase in income of 300% in real terms between 1970 and 1990, there was no corresponding increase in happiness [46]. Attempting to work around the diﬃculty in identifying a utility-optimal allocation of resources, Pareto proposed that bilateral exchanges may be classiﬁed into four types [5]: • Win - Win, in which case both parties’ utility increases; • Win - No Lose, in which case one party’s utility increases, and the other is no worse oﬀ; • Lose - No Lose, in which case one party’s utility decreases, and the other is no worse oﬀ; and • Win - Lose, in which case one party is better oﬀ and the other is worse oﬀ. 8 Pareto reasoned that by two individuals engaging in either a Win - Win or Win - No Lose exchange, the total utility in the system must have increased. In a Lose - No Lose exchange the total utility must have decreased, and in a Win - Lose exchange, we cannot say. Furthermore, Pareto noted that, assuming that individuals are economically rational (that is to say that they will only voluntarily enter into a trade which does not leave them any worse oﬀ), only Win - Win and Win - No Lose exchanges will take place. These became known as Pareto-superior trades. Through continued, voluntary Pareto-superior trades, the system drifts towards an equilibrium, at which point the only remaining trades possible require one party to be worse oﬀ [5]. However, as Beinhocker points out, this is not necessarily the globally optimum allocation of resources, the allocation which maximises the total utility in the system [5]. It is in fact only guaranteed to lead to a Pareto-optimal equilibrium, since it is quite possible that a Win - Lose trade will still exist, which will increase the total utility Of course, given the choice, the losing party in any remaining exchanges would not volunteer to make them. Interestingly, despite Smith’s claims of an invisible hand, Cliﬀ and Bruten note evidence that some market mechanisms, making use of simple, self-interested traders may stabilise to highly sub-optimal equilibria [16]. We have already seen how in some cases, stabilisation may not occur at all (see section 2.3). 2.3.3 Auctions Traditional economics proposes that the Pareto-optimal equilibrium point described may be reached through an auction mechanism; where autonomous actors repeatedly make those voluntary exchanges which increase their utility the most, facilitated by an auctioneer. In reality however, due to either imposed or unavoidable barriers to free exchange, additional costs associated with performing a transaction, or simply the cost associated with the time an auction may take, such Walrasian auctions are often an unrealistic ideal [5], and the behaviour described above may be more common. This is an issue which we will revisit. Many diﬀerent auction mechanisms exist in the real world, including the common English auction, found amongst other places on Ebay [21]; the Dutch auction; Vickrey auction and Continuous Double Auction, often used in ﬁnancial markets. Cliﬀ gives a useful introduction and critique of several auction mechanisms, including those listed above [14], while Friedman and Rust provide a more detailed look at the Continuous Double Auction [26]. In purely electronic markets, approaches to resource allocation make use of a range of auction mechanisms. In designing a mechanism, the aim is to encourage a high performing system overall, by making use of the self-interested nature of individual agents (as, for example in [50, 10, 20]). For many, the ultimate aim of such research is the automation of the mechanism design, appropriate to individual scenarios [37, 9, 18, 15]. Taking Cliﬀ’s work as an example of this, a parametrised mechanism design space is speciﬁed, which may be searched in 9 order to provide high performing mechanisms for speciﬁc scenarios [15]. Results from an evolutionary search demonstrate that classic, human-designed mechanisms are often far from optimal. 2.3.4 An Economy Of Agents Both service providers (sellers) and service users (buyers) may be represented in the marketplace by autonomous agents. Wooldridge and Jennings describe an agent as a being computer system in possession of (at least) autonomy, social ability, reactivity and pro-activeness [64]. More precisely, they specify an agent as being: • Clearly identiﬁable problem solving entities with well-deﬁned boundaries and interfaces; • Situated (embedded) in a particular environment they receive inputs related to the state of their environment through sensors and they act on the environment through eﬀectors; • Designed to fulﬁl a speciﬁc purpose they have particular objectives (goals) to achieve; • Autonomous they have control both over their internal state and over their own behaviour; • Capable of exhibiting ﬂexible problem solving behaviour in pursuit of their design objectives they need to be both reactive (able to respond in a timely fashion to changes that occur in their environment) and proactive (able to act in anticipation of future goals) [64] Multi-agent systems have become a dominant paradigm for the design of distributed systems, where individual components fulﬁl their own independent goals. Such systems are often characterised by the conﬂicting wishes of agents, who must enter into negotiation with each other in order to ﬁnd agreement. As we have seen, markets provide a mechanism by which individuals may settle on a particular agreement; in this case an allocation of resources. The same is true of negotiation in the wider sense, where voluntary concession is necessary amongst individuals in order to reach agreement [37]. A number of examples exist in the literature of agent-based market mechanisms being used to control the allocation of resources in various real-world scenarios. Jennings provides a helpful overview of the current state and direction of research in, inter alia, business to business electronic commerce [33]. Clearwater’s book, Market-Based Control: A Paradigm for Distributed Resource Allocation also provides a useful introduction to approaches to the use of computational markets in less expected scenarios, such as bandwidth allocation and air-conditioning control [13]. Other applications of computational markets include quality of service control in streaming media networks [54], call routing 10 allocation in communications networks [27, 28, 30, 31], the allocation of computational power in massively parallel internet computing [11], load control in intelligent networks [51] and in decision making for targeted advertising displays [56]. Clearly, the application areas are diverse. Despite the large number of examples of computational economies existing in the literature, there is an important clariﬁcation which must be made. A large proportion of this work, particularly the research directions laid out by Jennings [33], are concerned with the ability of agents to operate in a computational market operating within the real human economy. Continuous Double Auctions, for example, as studied inter alia by Vytelingum et al are of importance due to their use in many ﬁnancial markets around the world [61]. A further example is the quantity of research into intelligent trading agents, such as in [16], for use in on-line real-world trading systems. The markets in these approaches do not have a clear system-wide objective, since the system is itself the human economy. Of course, sections of the economy fulﬁl particular purposes, such as Amazon existing to ensure that readers, publishers and authors can allocate their books and DVD’s optimally [1]. A high volume of trading may even generate payoﬀs for market owners and brokers, but if the buyers’ and sellers’ payoﬀs remain high despite these purposes being achieved, the market cannot be said to have failed in its objective. However, recalling that in real markets, it may be impossible to place an absolute value on utility, in order to know whether individuals have indeed proﬁted from their interactions in the market, our only option may be to ask them. The alternative approach, as frequently used but less clearly delineated in the literature, is that of those examples contained in Clearwater’s book [13]. These systems make use of the theory of free trade, constructing artiﬁcial markets in order to achieve a system-wide objective. This objective is usually achieved by the market on behalf of the system owner. In these scenarios, the market is merely a distributed, possibly decentralised mechanism with which to control a system. Whilst much of the theory and many of the techniques employed are common to both approaches, it should be noted that there is a vital distinction to be made, which will drive the purpose of any research into computational economies. As we have seen, in a computational economy, the agents automate the interactions of the marketplace, though each has a very diﬀerent goal. The aim of a buying agent might be to secure the fastest and most reliable service at the lowest cost for its user. Conversely, a selling agent might aim to maximise the revenue for the service provider, or perhaps generate high levels of business. Whatever the business strategy of the service provider, the selling agent will be competing with similar agents from other providers for the same service users. Each agent will therefore have to employ its own strategy for success in the marketplace. 11 2.3.5 Decentralised, Not Just Distributed Auction-based markets however, suﬀer from one major drawback, which makes them less likely to be appropriate mechanisms for future service-oriented architectures: though they are distributed, they are not fully decentralised. As we discussed earlier, the Walrasian auction relies on a perfect auctioneer, and modern auction mechanisms have the same dependency. Indeed, in the human world, there are many examples of central ﬁgures in ﬁnancial and other markets; the New York Stock Exchange makes use of specialist, the NASDAQ a market maker and of course on-line auction sites such as Ebay are themselves auctioneers as well as facilitating the communication of individual traders’ reputations [21]. Cliﬀ and Bruten note that the vast majority of market-based control systems, particularly those described in [13], either rely on a central auctioneer, or else require some form of human intervention [16]. For these reasons, though much of the computation is done by individual agents and is therefore distributed, the system relies on a central node; the auctioneer. They argue that this leads to a brittleness of the system. However, a number of distributed auction mechanisms have also been proposed [25, 22, 41, 40, 30, 31, 32], which do not rely on one central auctioneer. These approaches reduce the fragility associated with reliance upon a single point, provide for a more scalable approach to electronic auctions and allow for dynamic composition of auction rooms. Typically, the central auctioneer is replaced by a number of local ones, which communicate through some secure means. At face value, this approach does not appear fully decentralised, though analysis of the degree of centralisation is currently lacking. Hausheer and Stiller present perhaps the most decentralised approach, relying on the use of spare, disinterested nodes in the network as auctioneers [32]. However, similarly to the vulnerabilities of the domain name system used in the internet [52], failures at certain points in the network may well cripple wider functionality, at best. In classic service-oriented architectures, reliant upon centralised UDDI registries and brokers [47, 24], this may in fact not be an issue, as distributed market mechanisms, posessing some degree of centralisation, overlay neatly onto the underlying network. Since brokers are central supernodes, they may also act as auctioneers. However, current research increasingly views service registries not as central supernodes, but as logical constructs, created from distributed information [49]. Risse and Knezevic describe one method by which this logical construction may be achieved, demonstrating its application in a decentralised, self-organising and service-based data store [55]. The desire to move towards decentralised architectures, which do not make use of central registries or brokers, requires a new, decentralised market-based approach. 2.3.6 Marketing Strategies The overhead of an auction, however, may not always be necessary. In Kuwabara et al’s equilibratory approach, the market settles on an equilibrium price for computational resources, when only the sellers set their prices [42]. Rather like in a supermarket, buyers then select products which maximise their own utility. 12 Their selling agents are not self-interested, however, instead calculating their oﬀer price based on their volume of sales in the previous round. Whilst in this example the market does ﬁnd equilibrium, this is not due to repeated voluntary Pareto-superior trades. However, Kuwabara et al’s approach shows the potential power of decentralised, auctioneer-free markets. In this approach, rather than employ trading strategies, which encode concession and other functions, agents must instead be able to set the price and quality of service oﬀered, based on information from the market and their own objectives. The sellers are, in fact, marketing their services, in the hope of attracting buyers. In business, McCarthy and Perreault’s classic text, Basic Marketing, describes the four P’s over which a good or service may be described, and go into what they call the marketing mix. Product, place, promotion and price must be blended into an integrated strategy to satisfy a target market [43]. Though the issues may be diﬀerent, exactly the same principle applies in computational economies. Each P is an issue upon which both buyer and seller must agree. The blending of (in this case) the four issues gives rise to a four-dimensional space, within which lies every possible market position. Furthermore, the issues are interrelated. A better quality product, for example, may well cost more to produce, in which case one would expect the price to increase also. More investment in promotion may well lead to a higher volume of sales, in which case the price (and hence proﬁt margin) could in turn be lowered. In heterogeneous markets, an optimal blend of issue values will exist for each target section of the market. A good strategy might identify a niche and adjust its blend of issues, in order to maximise the seller’s payoﬀ. Kephart et al analysed the dynamics of price-setting by sellers, studying the behaviour of brokers in a model of a heterogeneous electronic media market [39]. Their analysis explains a never-ending series of price wars, in which sellers would undercut each other in order to increase their payoﬀ. However, once the seller’s oﬀered price neared their marginal cost, rather than settle into equilibrium, it becomes rational for them to instead opt for a diﬀerent section of the market, providing a higher cost product, yet with a lower volume of sales. It is therefore vital to consider the dynamic nature of the market. Buyers and sellers may enter and leave the marketplace, and competitors or potential trading partners may alter their own strategies, as was demonstrated by Kephart et al. A high performing strategy at time t, may not reap the same rewards at time t + 1. Indeed, the maximal payoﬀ itself may change, making the success of a given strategy diﬃcult to measure. Agents must therefore be able to adapt to such changes in the levels of supply and demand, and should exploit niche opportunities to further achieve their 13 aims. Brooks et al describe the dilemma faced in strategy formulation succinctly in terms of the classic decision problem: Exploitation versus exploration [8]. 2.3.7 Trade Barriers Thusfar, we have neglected to give precise deﬁnition of a market. Begg et al deﬁne a market as “a set of arrangements by which buyers and sellers are in contact to exchange goods or services” [4] (Emphasis added) One crucial aspect of markets for resource allocation, is therefore the potential disconnection of individuals within a market. Geographic separation and the disconnectedness of individual traders, form barriers to free trade and will distort the market’s equilibria. Much of the literature concerning electronic auctions and marketing makes the assumption that the products provided by the sellers are accessible to all the buyers; that there is a level playing ﬁeld for competition. In reality, this will be far from the case. Beinhocker describes how, despite the theoretical predictions of economic equilibrium, this is rarely true in anything more than the simplest of cases [5]. There are many real-world factors, which may impact externally upon free trade, aﬀecting prices and creating trade barriers. Import taxes, distance-dependent shipping costs, cultural and language diﬀerences and transaction costs are but a few examples. Complete barriers to trading may also be brought about by local diﬀerences, legal prohibition, lack of telephone or Internet connectivity, the presence of sanctions and boycotts, or simply because a provider is too far away. Beinhocker argues that these factors act to keep the market not only from reaching equilibrium, but even from posessing global equilibrium price and issue values [5]. This eﬀect is likely to be highly important in the development of markets for computational services. Geographic separation, specialised local services and the presence of complex business arrangements between companies will lead to only partially connected networks of buyers and sellers, where costs may vary depending on ones position within the network. It is my belief that the study and design of computational markets for resource allocation in such scenarios remains largely unexplored. 2.4 An Evolutionary Approach Evolutionary computation provides us with the tools to be able to search for solutions to complex problems such as those faced by market agents. In the years since evolutionary computation was ﬁrst pioneered [34, 23, 53], a number of techniques have now matured, and have been applied to a wide range of computational intelligence problems, including search, learning, optimisation, design and game-theoretic analysis (see [66] and [68] for a comprehensive overview). 14 As we might therefore expect, there is more than one approach which could be employed here. 2.4.1 Understanding The Market Kephart et al’s analysis of price-wars demonstrates one analytical approach to understanding the dynamics of free markets [39]. Evolutionary game theory is another particularly useful tool with which to analyse this kind of competitive market scenario [63]. By allowing a population of strategies to evolve, rewarding the most successful, or ﬁttest, strategies by replicating them (potentially with some mutation), whilst culling the least ﬁt strategies, evolutionary dominance and stability of certain strategies may be explored. Vytelingum et al provide a useful introduction to the applicability of evolutionary game theory to the analysis of agent-based markets [62]. They detail how a two-population (buyers and sellers) approach demonstrates the dominance of certain strategies in mixed populations of traders in a continuous double auction. This approach appears highly applicable to other agent-based markets. In the more abstract sense, the interaction of selling agents in particular may be modelled with evolutionary games such as the Iterated Prisoner’s Dilemma (IPD) [3], its n-player version (NIPD) [67] and the Public Goods Game. In evolutionary games of this type, selﬁsh behaviour reaps short term rewards, but long term failure as opponents get the better of the situation. When a cooperative strategy is employed however, the players may exist in a Nash equilibrium [45]. That is to say that any unilateral action by any player will lead to that player being worse oﬀ in the long run. A game may have several (though incompatible) Nash equilibria and identifying them is often not simple. Indeed, given that all trade must be at least bilateral, and that agents are economically rational, a free market after the exhaustion of Pareto-superior trades (as described above) will exist in a Nash equilibrium. 2.4.2 Evolutionary Marketing Agents As previously discussed, evolutionary computation has been used successfully in the design of trading agents [16], and is also a prime candidate for use in developing intelligent marketing agents. There are a number of ways in which this may be achieved. Firstly, using an abstract model of the marketplace, expressed in terms of a game such as the Iterated Prisoners’ Dilemma [3], we might aim to search for strategies which perform well at that game. When the characteristics of the model are present in the real world, and the assumptions may be relied upon, such strategies may then be used on the problem proper. Evolutionary algorithms have shown promise in developing strategies for such games in the abstract sense (see for example the work in [3] and [67]). Beyond the abstract, there are examples in the literature of genetic algorithms being used to search for agent marketing strategies. Cheung et al used a 15 simpliﬁed model of the Australian consumer petroleum market to predict how sellers would modify their prices in a competitive environment [12]. They correctly replicated the implicit cartel behaviour observed in the real world, where despite a short-term temptation to cut their price in order to increase their market share, competing sellers implicitly cooperated to raise their prices in step. Despite explicit price-ﬁxing being illegal in most jurisdictions, the optimal game-theoretic Nash equilibrium is indeed to cooperate with competitors in order to avoid a crash in prices. Similarly, a variety of computational intelligence techniques, including genetic algorithms, have been used to attempt to learn and replicate the behaviour of human brand managers in the retail coﬀee market in North America [44]. Agent brand managers determined the occurrence and nature of various promotions and price discounts (two of the four P’s). The authors’ aim is to better understand the behaviour of human brand managers, so as to provide an insight into their real-world strategies. The agent strategies in both the above examples encode a function, which takes as its input a certain subset of information about the market’s state, perhaps including some bounded historical values. The agents process the state information according to the strategy encoded and perform some action as a result. In the former example, the action is an adjustment to the oﬀered price, given its price position in the marketplace relative to the other sellers. In the latter example, similar state information is used, though the actions are limited to certain real-world inspired actions, such as supermarket aisle displays, newspaper advertising and price discounting. Midgley et al used a market model named Casper, developed by Lee Cooper, one of the co-authors [44]. Casper aims to be representative of the real world in generating the market’s response to a given marketing strategy. One key diﬀerence between both the examples given and our problem is that we are not interested in modelling human behaviour, or in replicating it. Instead, we are concerned with the task of ﬁnding (and maintaining) an optimal marketing strategy, so as to maximise the payoﬀ of our trading agents and hence the eﬃciency of the system as a whole. The applicability of these approaches to our problem is far from guaranteed. Firstly, it is not suﬃcient simply to ﬁnd strategies which perform well in a particular marketplace, as we must be able to deal with the dynamic nature of the environment, as discussed above. In addition, we cannot assume that changes either to the levels of supply and demand, or to the strategies of the other agents will change in any predictable manner. Modelling all the possibilities is problematic. Therefore, we have either the option of working with a model of the market, (such as the Santa Fe model [2] or the Casper model, used in [44]), which embodies certain assumptions about it, or the diﬃculty of developing a strategy which either generalises or is capable of adapting to the unknown. Secondly, as the size of the marketplace increases and as the number of issues grows, the encoding needed to represent the state of the market will also 16 grow exponentially. For example, consider a market for a particular service, containing 10 sellers, each considering 3 issues (perhaps price, precision of the response and latency). As the authors themselves concede, extending Cheung et al’s strategy encoding for an agent in this marketplace would require each agent to encode actions for an impractically large number of states [12]. Another issue which presents is whether or not in fact a generalised optimal strategy exists, given the dynamic nature of the marketplace. Should the complexity of the strategies discussed here be extended further to consider changes in the market? Or, as mentioned above, would a model, assuming certain properties of the market be more appropriate? Finally, the latter option of developing evolutionary agents, capable of adapting to unknown environments appears to be a strong candidate to be considered. Though developed as a technique for use on stationary problems, evolutionary computation has, in recent years, been used in dynamic environments. In this scenario, the goal of the algorithm is no longer simply to ﬁnd the optimum, but to track its movement through the search space. Branke provides a comprehensive introduction to techniques which have been suggested for the modiﬁcation of traditional evolutionary algorithms to problems with dynamic ﬁtness landscapes [7]. The majority of approaches rely on the technique of retaining a certain amount of information about the landscape, through maintaining diversity in the population. A number of techniques have been proposed to ensure this, such as the periodic introduction of random immigrants to the population [29] and triggered hypermutation [17]. However, the random immigrants approach is a somewhat blunt instrument, which may disrupt the eﬀectiveness of the search and prevent the algorithm from ﬁnding the global optimum [57]. Triggered hypermutation may be highly eﬀective, but relies on certain domainspeciﬁc knowledge, such as what should be considered a trigger. A third option, when some pattern exists in the nature of the environment’s dynamics, is the retention of knowledge of the landscape in the individuals themselves; a form of memory [6, 57]. 17 Chapter 3 Statement Of Work To Be Done 3.1 The Problem Area The key question I will aim to answer is that of how, given a decentralised, dynamic service-based environment, we might ensure an eﬀective allocation of resources, to fulﬁl the needs of service users and provide overall quality of service and eﬃciency. Furthermore, since many architectures may involve partially connected and/or geographically separated networks of both providers and users, how might this aﬀect the existence of optimal equilibria and the market’s ability to ﬁnd them? Arguably, the problem remains interesting in distributed, yet centralised markets, such those which use the Continuous Double Auction. However, focusing on decentralised service-oriented architectures will ensure that this work remains relevant to the next generation of architectures, which will be built without central nodes acting as registries or brokers [49]. Decentralisation will allow for highly dynamic networks of providers and users to form, join, divide and dissipate. Understanding the properties of markets operating in such networks will be vital to their ability to work eﬃciently for their constituent individuals. Furthermore, this understanding will inform the design of intelligent market agents, able to operate in such scenarios. It is hoped that this work will therefore make a useful contribution to the development of decentralised, market-based serviceoriented architectures. 3.2 The Work Ahead In order to address the problem, work may be divided into two primary tasks. 1. To investigate, using evolutionary game theory and other analytical techniques, the properties and dynamics of an agent-based free market in various connected, disconnected, partially connected and geographically separated states. 18 2. To develop, using evolutionary computation techniques, agents capable of acting on behalf of service providers and users. Furthermore, to analyse the applicability, performance and weaknesses of the approach developed, with a view to both malicious and irrational behaviour. The remainder of this chapter will discuss each task in turn, and how the signiﬁcance of the work may be evaluated. 3.3 Investigation As with any research topic, a sound analysis of the problem in question is a vital prerequisite to developing and analysing solutions. With this in mind, I will ﬁrst aim to investigate the properties and dynamics of an agent-based free market in various states of connectedness and geographical separation. 3.3.1 Problem Speciﬁcation In the problem’s most general form, we may describe a service class, C. We then have a set of buyers, B, who act on behalf of service users to purchase resources of type C from a set of sellers, S. In this scenario, a market economy exists for service class C. We call this economy EC . The economy may be represented as a graph, with both buying and selling agents as nodes, and the ability for two agents to trade represented as links between the nodes (see ﬁgure 3.1). Using notation from graph theory, the economy, EC may therefore be described as: EC = (B ∪ S, T ) (3.1) where the set of edges, T is the set of all available trade routes. The presence of a trade route (a link between an element of B and an element of S) means that there is no absolute barrier to the two trading: they are in the same market. More formally, we deﬁne a market for C, MC to be any wholly contained subgraph of EC , such that all nodes connected to every node in MC are also in MC . Given this deﬁnition, we observe that the majority of scenarios considered in the literature review make use of the simplest of its possible scenarios; an economy consisting of one market, where all buyers and sellers are fully connected (i.e. T is the set of links which connect all of B to all of S), therefore each may trade with any other agent (see ﬁgure 3.1). For simplicity at this stage, we assume that the market operates in discrete time steps. During one time step, each buyer b ∈ B, may purchase a quantity, Qb , of the service provided by seller s ∈ S. Recalling the idea that these s purchases are voluntary bilateral exchanges, at the completion of each time step, an agent’s instantaneous payoﬀ may be determined by the value it associates with what it has gained, subtracted by the value it associates with what it has given away. In a system with two goods, for example where buying agents may 19 Figure 3.1: A fully connected market exchange money, p for a seller’s bandwidth, q, the payoﬀ for agent i, at time t, may be denoted by: Pi (t) = Ui (pi (t), qi (t)) (3.2) where Ui is agent i’s utility function, and returns a real value, determined by the agent’s preference between the amounts of each good given (in this case p and q). For buying agents, the function might be balanced to provide higher payoﬀ values when the desired amount of bandwidth has been obtained at a low cost. For some buyers, the function might degrade gracefully as the price increases, or as the bandwidth obtained reduces. For others, the payoﬀ may drop dramatically if the price exceeds an upper limit. Of course, this will depend on the purpose and wishes of the buyer, and as described above, will generally not be known. The same will be true of sellers. Each will have a reserve price for their service, based on their own costs, and those may well be tied to other markets. Through continued voluntary Pareto-superior exchanges between buyers and sellers, it is expected that resources become allocated in a Pareto-optimal conﬁguration. More formally, the total utility in the system at time t, |B| |S| U (t) = i=1 Ui (pi (t), qi (t)) + j=1 Uj (pj (t), qj (t)) (3.3) 20 should increase with respect to t. 3.3.2 An Evolutionary Investigative Approach Now, consider a static market, with a ﬁxed quantity of a given good, where agents do not come and go, nor do their strategies or utility functions change over time. Through continued voluntary trading, we expect a market of this type to reach an equilibrium value for price and other issues. This hypothesis may be tested using an evolutionary approach. Sellers’ oﬀer prices (and other issue values) may be encoded as candidate solutions in an evolutionary algorithm, and evolved, using the market’s response to their oﬀers as a ﬁtness function. Their payoﬀ, as described above, will determine the ﬁtness of each oﬀer. By applying evolutionary operators to the sellers’ oﬀers, it is expected that the evolutionary search will converge to the equilibrium point; the point at which their payoﬀ is maximised. The market’s response to an oﬀer may be calculated in one of a number of ways. Firstly, a model of a market may be constructed, giving an expected payoﬀ for a given oﬀer. This type of approach was used by Cheung et al [12] and Midgley et al [44], though their models were constructed to model the speciﬁc markets in which they were interested and are not generically applicable. An alternative approach is to model the buyers also as autonomous agents. The traditional theory of free markets described above assumes that buyers behave in a rational, utility maximising manner, considering all available options [5]. In simulation, this is not a diﬃcult behaviour to achieve for buying agents, at least when considering only immediate payoﬀs. By selecting that seller which maximises its own payoﬀ, a population of buyers will provide the necessary payoﬀ information to the evolving sellers. However, buyers (and sellers) are often unable or unwilling to behave according to the rules of economic rationality [38]. Some buyers may attribute a higher value to the concept of spread buying, in which products are selected from a range of sellers in order to minimise the risk associated with a seller reneging on a deal. A broad portfolio of purchases, as opposed to putting all ones eggs in the same basket, despite not being strictly utility maximising, forms a more robust approach in the presence of an unreliable or untrustworthy market. Other reductions in buyer rationality include arbitrary individual preferences, social factors and bounds on available information and decision-making power [5]. Despite the relevance and highly interesting nature of these factors, suspensions of rationality are themselves a topic for study, and beyond the scope of this project. It is with this caveat in mind, however, that we investigate the properties of the markets in question with rational assumptions in place. Using an evolutionary simulation of this type, the equilibrium values in our simple market scenario may be observed, and this may be achieved for a variety of conﬁgurations of market. 21 Hypothesis 1: In a static, fully connected market, with a ﬁxed quantity of a good, an equilibrium value for each issue of that good exists. Hypothesis 2: By evolving selling agents’ issue values with respect to their payoﬀ, the sellers’ issue values will converge to their equilibria. 3.3.3 Partially Connected Markets Our stated aim at this stage is to understand the presence and position of equilibria in partially connected networks of agents. Consider a set of sellers, S and a set of buyers, B, arranged into two separate isolated markets (subgraphs 1 2 of EC ), MC and MC as shown in ﬁgure 3.2a. Each market will have its own equilibrium, which may be found using the evolutionary method above. Figure 3.2: Two markets, isolated (a) then partially connected (b) 1 2 Consider further, that seller si from MC expands somewhat into MC , adding a connection between itself and buyer bj , as depicted in ﬁgure 3.2b. Presuming 1 that si is selling at the equilibrium price for MC , depending on the equilibrium 2 price of MC , bj will either be able to consider an oﬀer less favourable than its current options, or else avail itself of a much better price. This itself will 1 2 have a knock-on eﬀect. If MC ’s equilibrium price was less than MC ’s, then bj will be able to increase its payoﬀ by switching from its current provider to 22 2 si . The payoﬀ for the other sellers in MC will therefore suﬀer. If however, 1 2 MC ’s equilibrium price was higher than MC ’s, then even though it is not in bj ’s interest to immediately switch to si , si may well no longer be trading at its optimal price. By adapting its oﬀer to take advantage of the new potential 2 custom, competition will increase in MC , whilst of course also aﬀecting the 1 supply available in MC . Clearly, this informal reasoning illustrates the range of complex interrelationships brought about simply by adding one link between two otherwise distinct sub-graphs. In more complex markets, trade barriers mean that networks may be arranged into any partially connected topology and furthermore may be subject to change over time. Similar to the model used in [39], multiple optima may exist within the payoﬀ landscape, meaning that there may be no single equilibrium price. How do equilibratory free market mechanisms such as that in [42] operate in such environments? Being able to understand the development of equilibria in this type of market will be highly useful to those aiming operate successfully in them. Hypothesis 3: A modiﬁcation of the static market model, such that it is no longer fully connected, possesses diﬀerent optimal issue values from its fully connected counterpart. Hypothesis 4: Such partially connected markets necessarily have multiple peaks in their payoﬀ landscapes, and therefore encourage price wars rather than equilibrium. This may be demonstrated using the evolutionary approach. 3.3.4 Geographically Separated Markets A more subtle, but more likely scenario than that described above, is that particular trade routes are not unavailable, but have associated with them additional transaction costs. Examples of this might be increased delivery charges associated with shipping a product further from base, bank charges or tariﬀs incurred by international transfers and currency exchanges, or requiring the laying of new communication channels to currently unserved locations. These costs may act to distort the equilibrium, or prevent the market from reaching it [5]. These additional geographically-determined costs may be introduced easily into our graph-based model, by weighting the links between agents. Recall that a good may be described over any number of issues, including price, but also such as quality, latency, accuracy or size. Each weighted link may add to or subtract from the issues of a good being transferred along it. For example, suppose two buyers independently buy the same book on-line, for the same price, both to be delivered by post. Since both buyers and the seller are on the postal network, links exist between them in the economy graph; they are in the same market. However, whilst delivery to the ﬁrst buyer in Birmingham takes only until the following morning, the postal service takes three days to ship the book to the second buyer’s location in the Orkney Isles. 23 The links between the buyers and the seller are weighted diﬀerently for the delivery time issue. Both buyers receive the same good, the book they requested, however the delivery time issue over which the book is sold varies between them, as a result of the weighted connection. Depending on the buyers’ preferences, this variation may aﬀect the utility to be gained from the trade. The more remote buyer may prefer, in this instance, to take only two days to travel to the mainland in order to purchase the book in a High Street shop. The on-line bookstore, as a result of its use of the postal service, is subject to more heavily weighted links in certain sections of the market. As a result of this, they may be inclined to alter their price in order not to sacriﬁce a proportion of the aﬀected market segment. The equilibrium therefore, must have moved. Similarly to in the partially connected scenario, this simple, informally described example demonstrates how alterations in the connectedness of the market may have an eﬀect upon the equilibrium issue values to be achieved. Indeed, in all circumstances of these kinds, is an equilibrium necessarily guaranteed to exist? If not, what circumstances might be responsible for bringing the equilibrium about? Hypothesis 5: That the introduction of issue weightings to the links in a fully connected market graph leads to diﬀerent optimal issue values than in an unweighted graph. Hypothesis 6: As with partially connected markets, issue weighting leads to multiple peaks in the payoﬀ landscape, and therefore encourages price wars rather than equilibrium. This may also be demonstrated using the evolutionary approach. 3.3.5 Outcomes In order to successfully carry out this investigation into partially connected and geographically separated markets and test the hypotheses detailed above, I aim to complete the following objectives. • Specify in detail, and with reasoning, a number of connected, partially connected and geographically separated market scenarios, whose behaviour is to be investigated; • Produce an implementation of the above described evolutionary investigative model; • Use the implementation to produce results concerning the scenarios speciﬁed; • Analyse the results, producing a robust statistical commentary and analysis of the payoﬀ landscapes. 24 The above implementation will be produced in a manner such that the results are easily demonstrable to interested parties, such as BT and collaborating partners on the EPSRC Market-Based Control project. This may include visualisations and other exploratory or interactive functionality. 3.4 Development Having completed an analysis and gained an understanding of the properties of partially connected and geographically separated service-based computational markets, I aim to use this knowledge to develop agents capable of marketing (setting issue values) on behalf of service providers and users. Depending on the overall aim of the system, if indeed one is present, agents may be designed with a number of individual objectives in mind. For example, in a real market, the agents’ aim might be to maximise their payoﬀ from their engagement with the wider system. Conversely, in a market which we wholly own, there may be an imposed system-wide goal. For example, as in [42], we may wish to achieve a speciﬁc desired allocation (e.g. even load balancing). 3.4.1 Initial Experiments Initial experiments have been carried out, replicating the equilibratory approach in a fully connected market. Early results suggest that the load balance achieved by Kuwabara et al [42] may be controlled and adjusted by varying individual agents’ utility functions. However, it is as yet unclear how generalised or robust this behaviour is. In other early experiments, the sellers’ heuristic strategy (see section 2.3.6 for details) was replaced with a population-based evolutionary algorithm, in which the price was encoded in each seller. Sellers were then co-evolved against each other and a population of buyers. Early results suggest that the evolutionary algorithm performs well at ﬁnding a good balance of issue values for a given target market, and that eﬃcient load balancing can be achieved using this approach, though results were not as favourable as with the original heuristic. However, when the service is described over more than one issue, the usage information relied on by the heuristic must be decomposed into each issue, and hence the simple approach used in the heuristic is not possible. The evolutionary algorithm is however able to search a multi-issue product space, and similar results to the evolutionary approach with one issue (just price) were achieved. The initial results of these experiments were presented at the national EPSRC Market-Based Control project meeting in May and are currently being written up. It is clear however, that further analysis is necessary. 3.4.2 Going Forward Given either the heuristic-based equilibratory approach proposed by Kuwabara et al [42], or an evolutionary variant of this, it will be interesting to observe 25 the behaviour of such strategies in the partially connected and geographically separated markets described above. Can, for example, such strategies ﬁnd the equilibria (if one exists) determined by the investigations described above (see section 3.3)? If there is no such equilibrium, can evolutionary agents obtain an optimal set of issue values, or do they suﬀer from the same myopic price wars as described by Kephart et al [39]? A key aim will be to determine the extent to which modiﬁcations to agent strategies might be necessary in such scenarios, such as their ability to resist such myopic behaviour. Rather than using a population of buyers, agent strategies may also be developed against a market model, as in many of the examples described in section 2.4.2. Rather than focusing on the generalised situation, if we are instead able to make certain assumptions about the operation of the market, it may be possible to ﬁnd agent strategies which perform well in these scenarios. Whether or not this is a feasible option, and which model should be chosen, will be determined by further research into markets in the application domain, such as currently existing service-based markets. Hypothesis 7: Kuwabara et al’s heuristic-based equilibratory agents are incapable of achieving equilibratory issue values in the presence of multiple issues over which a good is described. Hypothesis 8: A population of evolutionary agents, capable of learning to decompose payoﬀ information into multiple issues are able to achieve equilibrium issue values in the presence of multiple issues. Hypothesis 9: In partially connected and/or geographically separated markets posessing multiple peaks in the payoﬀ landscape, Kuwabara et al’s heuristic-based strategies [42] suﬀer from the myopia described by Kephart et al [39] and therefore produce price-wars. Hypothesis 10: These price wars may be avoided or minimised by the use of evolutionary agents capable of acting with less myopic behaviour. As with the previous task, there are a number of outcomes which I will aim to achieve in order to test the stated hypotheses. • Specify an evolutionary alternative Kuwabara et al’s price-setting heuristic, capable of acting over multiple issues, and complete a comparative analysis of the behaviour of each; • Investigate the ability of the agent strategies to reach any equilibria or optimality determined in the previous section for partially connected and geographically separated markets; • If appropriate, determine a market model to be used for comparison with the above experiments; • Produce analysis of and comment on the applicability, performance and weaknesses of the strategies developed, with a view to both malicious and irrational behaviour. 26 Similarly to in the previous section, it is anticipated that demonstrable simulations will be produced, to aid in the presentation of the work to other interested parties. 3.5 Evaluation Of The Work The work to be produced during this project should, of course, be evaluated with equal rigour to that which is applied to all other academic work. It is with this is mind that the above objectives will be tackled. Apart from a self-critical eye, opportunities for presenting early results and obtaining feedback will be seized. So far, the following opportunities both within the school and further aﬁeld have been identiﬁed as available: • Weekly supervisory / local Market-Based Control meetings; • Natural Computation Talk meetings; • Regular national EPSRC Market-Based Control project meetings (every three months); • Visits and presentations to BT; • Thesis Group meetings; • Cake Talks and other departmental series; • Adhoc poster presentation sessions; • Summer schools; • Conferences and workshops. With a view to the ﬁnal point, I hope to be able to produce results-oriented write-ups of current work roughly every six-months. The presence of such deadlines, tied to the objectives detailed in the previous section, will help to keep pace. This schedule will, of course, be partially determined by conference and other deadlines, as I hope to be able to make regular submissions. Apart from the obvious beneﬁt of having work published, this is an ideal opportunity to receive signiﬁcantly wider feedback and help to determine future directions. I believe that the completion of the tasks outlined above will result in a new contribution to current understanding of the behaviour of partially connected and geographically separated markets. This understanding will be of practical use in clarifying and deciding upon the appropriateness of diﬀerent forms of computational markets (e.g. auction-based, equilibratory) for given problems. Furthermore, this knowledge will be useful in informing the development of agent marketing or price-setting strategies, speciﬁc to various scenarios. More speciﬁcally, and in addition to the robust testing of the hypotheses detailed above, achievement of the following criteria may be used to measure success. 27 Table 3.1: Success Criteria Criteria Demonstration of how evolutionary methods may be used to analyse and show the equilibria and dynamics of free markets. Contribution to the understanding of computational market equilibria and dynamics, particularly in partially connected and geographically separated scenarios. Development of selling agents, capable of achieving performance comparable with Kuwabara et al’s heuristic-based sellers, but for multiple issues. Development of selling agents, capable of achieving optimal issue values and producing a similar high performance in partially connected and geographically separated markets. Where markets posses more dynamic (e.g. price-war) behaviour, the development of selling agents which are less susceptible to such price-wars. Success Measure Demonstration, statistical results, validation against economic theory. Published peer-reviewed paper(s). Statistical results. Statistical results. Statistical signiﬁcance when compared with heuristic-based agents. In addition to the research objectives already laid out, a key evaluation process will be to identify problems left unsolved by the work. These will be detailed, along with potential approaches to solving them. 3.6 Timetable Given the unpredictable nature of research, any timetable will be at best a target and estimate of the work to come. With this in mind, the following table is a proposed plan for the remainder of this research. The asterisks denote the expectation that results produced should be suitable for submission for publication. The hypotheses referred to are described in section 3.3. 28 Table 3.2: Research Timetable Phase 1: Completing and writing up initial experiments. Date Activity September - Complete initial experiments and analysis of comparison of heuristic October 2007 and evolutionary agents in the equilibratory mechanism, with a view to a load balancing objective as in [42]. Hypotheses 1 and 2. * Phase 2: Evolutionary investigation of markets. Date Activity November 2007 Specify, with reasoning, connected, partially connected and geographically separated market scenarios for investigation. December 2007 Specify, design and implement the evolutionary model with which to - January 2008 investigate the markets speciﬁed, with reference to [62] February 2008 Specify and run experiments, in order to obtain results on the pres- March 2008 ence and conditions for ideal equilibria in the markets speciﬁed. Hypotheses 3, 4, 5 and 6. April - May Following from results obtained, further specify and investigate sce2008 narios of interest. Thesis Group Report 4 due. June 2008 Write up results and analysis of experiments. * Phase 3: Agent design and experimentation. Activity Based on earlier initial experiments, make use of knowledge gained from investigations into partially connected and geographically separated markets to specify and design evolutionary marketing agents capable of high performance in such markets. September - Implement evolutionary agents and test on a wide variety of marNovember 2008 ket scenarios, possibly including populations of agent buyers, market models and real-world service-oriented architectures. (Depending on the scope of testing available, this may require further time.) Thesis Group Report 5 due. December 2008 Analyse results of above experiments and produce detailed commen- January 2009 tary on the abilities, applicability and weaknesses of the agents developed, in comparison with idealised results obtained in phase 2. Hypotheses 7, 8, 9 and 10. * February 2009 Suggest, implement and test modiﬁcations. Date July - August 2008 Phase 4: Evaluation and writing up. Activity Review work completed so far, including a rigorous evaluation of the applicability and signiﬁcance of the results and contribution made to knowledge, with reference to feedback from the numerous opportunities mentioned in section 3.5. Based on this evaluation, determine if further work is required. This collation and review of results will also begin the writing up phase. Thesis Group Report 6 due. Writing up. Thesis Group Report 7 due (October). Contingency time. Thesis Group Report 8 due (April). Date March - April 2009 May 2009 - Nov 2009 December 2009 - May 2010 29 Bibliography [1] Amazon. http://www.amazon.com/. [2] W. Brian Arthur. Complexity in economic and ﬁnancial markets. Complexity, 1(1), April 1995. [3] Robert Axelrod. The Evolution of Cooperation. Basic Books, New York, 1984. [4] David K.H. Begg, Stanley Fischer, and Rudiger Dornbusch. Economics. McGraw-Hill, London, UK, 7th edition, 2002. [5] Eric D. Beinhocker. The Origin of Wealth. Random House Business Books, London, UK, 2007. [6] J¨rgen Branke. Memory enhanced evolutionary algorithms for changing opu timization problems. In Proceedings of the IEEE Congress on Evolutionary Computation, 1999. [7] J¨rgen Branke. u Kluwer, 2001. Evolutionary Optimization in Dynamic Environments. [8] Christopher H. Brooks, Robert S. Gazzale, Rajarshi Das, Jeﬀery O. Kephart, Jeﬀrey K. MacKie-Mason, and Edmund H. Durfee. Model selection in an information economy: Choosing what to learn. Computational Intelligence, 18(4):566–582, 2002. [9] Andrew Byde. Applying evolutionary game theory to auction mechanism design. In IEEE International Conference on E-Commerce, pages 347–354, 2003. [10] Andrew Byde. A comparison between mechanisms for sequential compute resource auctions. In Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multi-agent Systems, Hakodate, Japan, 2006. [11] Peter Cappello, Bernd O. Christiansen, Michael O. Neary, and Klaus E. Schauser. Market-based massively parallel internet computing. In Proceedings of the 3rd Working Conference on Massively Parallel Programming Models, pages 118–129, Washington, DC, USA, 1997. IEEE Computer Society. 30 [12] Yen Cheung, Sue Bedingﬁeld, and Stephen Huxford. Monitoring and interpreting evolved behaviours in an oligopoly. In Proceedings of the IEEE International Conference on Evolutionary Computation, pages 697–701, 1997. [13] Scott H. Clearwater, editor. Market-Based Control: A Paradigm for Distributed Resource Allocation. World Scientiﬁc, Singapore, 1996. [14] Dave Cliﬀ. Minimal-intelligence agents for bargaining behaviors in marketbased environments. Technical Report HPL-97-91, HP Labs, Bristol, UK, 1997. [15] Dave Cliﬀ. Explorations in evolutionary design of online auction market mechanisms. Journal of Electronic Commerce Research and Applications, 2003. [16] Dave Cliﬀ and Janet Bruten. Simple bargaining agents for decentralized market-based control. Technical Report HPL-98-17, HP Laboratories, Bristol, UK, 1998. [17] H. G. Cobb. An investigation into he use of hypermutation as an adaptive operator in genetic algorithms having continious, time-dependent nonstationary environments. Technical Report Memorandum Report 6760, Navy Research Lab, 1990. [18] Vincent Conitzer and Tuomas Sandholm. Self-interested automated mechanism design and implications for optimal combinatorial auctions. In Proceedings of the 5th ACM Conference on Electronic Commerce, pages 132– 141, New York, NY, USA, 2004. ACM Press. [19] Ellen T. Curtiss and Susan Eustis. Services oriented architecture: Market opportunities, strategies, and forecasts, 2006 to 2012. Report SH29821464. Available at http://www.wintergreenresearch.com/reports/soa.html, 2006. [20] Esther David, Alex Rogers, Nicholas R. Jennings, Jeremy Schiﬀ, Sarit Kraus, and Michael H. Rothkopf. Optimal design of english auctions with discrete bid levels. ACM Transactions on Internet Technology, 7(2):Article 12, 2007. [21] Ebay. http://www.ebay.com/. [22] Marc Esteva and Julian Padget. Auctions without auctioneers: Distributed auction protocols. In Lecture Notes in Artiﬁcial Intelligence, volume 1788, chapter Agent-mediated Electronic Commerce II, pages 20–28. SpringerVerlag, Berlin, Germany, 2000. [23] Lawrence J. Fogel. Artiﬁcial Intelligence Through Simulated Evolution. John Wiley and Sons Inc, New York, NY, USA, 1966. [24] Organization for the Advancement of Structured Information Standards. Universal description, discovery and integration. http://www.uddi.org/. [25] M. K. Franklin and M. K. Reiter. The design and implementation of a secure auction service. In Proceedings of the 1995 IEEE Symposium on Security and Privacy, page 2, Washington, DC, USA, 1995. IEEE Computer Society. 31 [26] Daniel P. Friedman and John Rust. The Double Auction Market: Institutions, Theories, and Evidence. Westview Press, Boulder, Colorado, USA, 1993. [27] M. A. Gibney and Nick Jennings. Dynamic resource allocation by marketbased routing in telecommunications networks. In Proceedings of 2nd International Workshop on Multi-Agent Systems and Telecommunications, pages 102–117, Paris, France, 1998. [28] M. A. Gibney, Nick R. Jennings, N. J. Vriend, and J. M. Griﬃths. Marketbased call routing in telecommunications networks using adaptive pricing and real bidding. In Proceedings of the 3rd International Workshop on Multi-Agent Systems and Telecommunications, pages 50–65, Berlin, Germany, 1999. Springer. [29] John Greﬀenstette. Genetic algorithms for changing environments. In Proceedings of the Second International Conference on Parallel Problem solving from nature, pages 137–144, Amsterdam, The Netherlands, 1992. Elsevier. [30] Nadim Haque, Nick R. Jennings, and Luc Moreau. Resource allocation in communication networks using market-based agents. International Journal of Knowledge Based Systems, 18(4-5):163–170, 2005. [31] Nadim Haque, Nick R. Jennings, and Luc Moreau. Scalability and robustness of a network resource allocation system using market-based agents. Netnomics, 7(2):69–96, 2005. [32] David Hausheer and Burkhard Stiller. Peermart: The technology for a distributed auction-based market for peer-to-peer services. In Proceedings of the IEEE International Conference on Communications, volume 3, pages 1583–1587, 2005. http://www.peermart.net/. [33] M. He, Nick R. Jennings, and H. Leung. On agent-mediated electronic commerce. IEEE Transactions on Knowledge and Data Engineering, 15(4):985– 1003, 2003. [34] John Henry Holland. Adaptation in natural and artiﬁcial systems : an introductory analysis with applications to biology, control, and artiﬁcial intelligence. MIT Press, 1975. [35] Michael Huhns and Munindar P. Singh et al. Research directions for serviceoriented multiagent systems. IEEE Internet Computing, 9(6):65–70, 2005. [36] Michael Huhns and Munindar P. Singh. Service-oriented computing: Key concepts and principles. IEEE Internet Computing, 9(1):75–81, 2005. [37] Nick R. Jennings, P. Faratin, A. R. Lomuscio, S. Parsons, C. Sierra, and M. Wooldridge. Automated negotiation: Prospects, methods and challenges. International Journal of Group Decision and Negotiation, 10(2):199–215, 2001. [38] Daniel Kahneman, Paul Slovic, and Amos Tversky, editors. Judgment under Uncertainty: Heuristics and Biases. Cambridge University Press, Cambridge, UK, 1982. 32 [39] Jeﬀrey O. Kephart, James E. Hanson, and Jakka Sairamesh. Price-war dynamics in a free-market economy of software agents. In Proceedings of the Sixth International Conference on Artiﬁcial Life, pages 53–62, Cambridge, MA, USA, 1998. MIT Press. [40] Hiroaki Kikuchi. (m+1)st-price auction protocol. In Proceedings of the 5th International Conference on Financial Cryptography, pages 351–363, London, UK, 2002. Springer-Verlag. [41] Hiroaki Kikuchi, Shinji Hotta, Kensuke Abe, and Shohachiro Nakanishi. Distributed auction servers resolving winner and winning bid without revealing privacy of bids. In Proceedings of the Seventh International Conference on Parallel and Distributed Systems: Workshops, page 307, Washington, DC, USA, 2000. IEEE Computer Society. [42] Kazuhiro Kuwabara, Toru Ishida, Yoshiyasu Nishibe, and Tatsuya Suda. An equilibratory market-based approach for distributed resource allocation and its applications to communication network control. In Scott H. Clearwater, editor, Market-Based Control: A Paradigm for Distributed Resource Allocation, pages 53–73. World Scientiﬁc, Singapore, 1996. [43] E. Jerome McCarthy and William D. Perreault. Basic Marketing. Irwin, Homewood, IL, USA, 8th edition, 1984. [44] David F. Midgley, Robert E. Marks, and Lee G. Cooper. Breeding competitive strategies. Management Science, 43(3):257–275, 1997. [45] John Nash. Equilibrium points in n-person games. In Proceedings of the National Academy of the USA, volume 36, pages 48–49, 1950. [46] Daniel Nettle. Happiness: The Science Behind Your Smile. Oxford University Press, Oxford, UK, 2005. [47] Eric Newcomer. Understanding Web Services: XML, WSDL, SOAP and UDDI. Addison-Wesley, Indianapolis, Indiana, USA, May 2002. [48] M. P. Papazoglou and D. Georgakopoulos. Service-oriented computing special section: Introduction. Communications of the ACM, 46(10):24–28, 2003. [49] Michael P. Papazoglou, Paolo Traverso, Schahram Dustdar, and Frank Leymann. Service-oriented computing: Research roadmap. 2006. [50] Steve Phelps, S. Parsons, and Peter McBurney. An evolutionary gametheoretic comparision of two double-auction market designs. In Proceedings of the 6th Workshop on Agent Mediated Electronic Commerce, New York, NY, USA, 2004. [51] K. Prouskas, A. Patel, J. Pitt, and J. Barria. A multi-agent system for intelligent network load control using a market-based approach. In Proceedings of the Fourth International Conference on MultiAgent Systems, pages 231–238, Washington, DC, USA, 2000. IEEE Computer Society. 33 [52] Venugopalan Ramasubramanian and Emin Gun Sirer. Perils of transitive trust in the domain name system. Technical Report TR2005-1994, Cornell University, Ithaca, New York, USA, 2005. [53] I. Rechenberg. Evolutionsstrategien: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Frommann-Holzboog, Stuttgart, Germany, 1973. [54] D. Reininger, D. Raychaudhuri, and M. Ott. Market-based bandwidth allocation policies for qos control in broadband networks. In Proceedings of the First International Conference on Information and Computation Economies, pages 101–110, New York, NY, USA, 1998. [55] Thomas Risse and Predrag Knezevic. A self-organizing data store for large scale distributed infrastructures. In Proceedings of the 21st International Conference on Data Engineering Workshops, page 1211, Washington, DC, USA, 2005. IEEE Computer Society. [56] Alex Rogers, Esther David, T. R. Payne, and Nick R. Jennings. An advanced bidding agent for advertisement selection on public displays. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multi-agent Systems, 2007. [57] Philip Rohlfshagen and John A. Bullinaria. Alternative splicing in evolutionary computation: Adaptation in dynamic environments. In Proceedings of the IEEE Congress on Evolutionary Computation, 2006. [58] Munindar P. Singh and Michael N. Huhns. Service-Oriented Computing: Semantics, Processes, Agents. John Wiley and Sons, 2005. [59] Adam Smith. The Wealth of Nations. Everyman’s Library, reprint edition, 1991. [60] Ken Steiglitz, Michael L. Honig, and Leonard M. Cohen. A computational market model based on individual action. In Scott H. Clearwater, editor, Market-Based Control: A Paradigm for Distributed Resource Allocation, chapter 1, pages 1–27. World Scientiﬁc, 1996. [61] Perukrishnen Vytelingum, Dave Cliﬀ, and Nick R. Jennings. Evolutionary stability of behavioural types in the continuous double auction. In Proceedings of the 8th International Workshop on Agent-Mediated Electronic Commerce, 2006. [62] Perukrishnen Vytelingum, Dave Cliﬀ, and Nick R. Jennings. Analysing buyers and sellers strategic interactions in marketplaces: An evolutionary game theoretic approach. In Proceedings of the 9th International Workshop on Agent-Mediated Electronic Commerce, pages 141–154, Hawaii, USA, 2007. [63] E. Roy Weintraub. Conﬂict and Cooperation in Economics. MacMillan Studies in Economics. MacMillan, London, 1975. [64] Michael Wooldridge and Nicholas R. Jennings. Intelligent agents: Theory and practice. Knowledge Engineering Review, 10(2):115–152, 1995. 34 [65] Jian Yang. Web service componentization. Communications of the ACM, 46(10):35–40, 2003. [66] Xin Yao. An overview of evolutionary computation. Chinese Journal of Advanced Software Research, 3(1):12–29, 1996. [67] Xin Yao and Paul J. Darwen. An experimental study of n-person iterated prisoner’s dilemma games. In Selected papers from the AI’93 and AI’94 Workshops on Evolutionary Computation, Process in Evolutionary Computation, pages 90–108, London, UK, 1995. Springer-Verlag. [68] Xin Yao and Y. Xu. Recent advances in evolutionary computation. Journal of Computer Science and Technology, 21(1):1–18, January 2006. 35

Autonomic Solutions for Virtualised ICT Systems more

Log In

Reset Password