Microsoft Word - Disadvantaged Student Transportation 20210301_mm.docx

DIVISION OF THE HUMANITIES AND SOCIAL SCIENCES

CALIFORNIA INSTITUTE OF TECHNOLOGY

PASADENA, CALIFORNIA 91125

Using Theory, Markets and Experimental Methods to Improve a

Complex Administrative Decision

Process

: School

Transportation for

Disadvantaged Students

Charles R. Plott (California Institute of Technology)

Gary Stoneham (Centre for Market Design, University of Melbourne)

Hsing

Yang Lee (California Institute of Technology)

Travis Maron (California

Institute of Technology)

SOCIAL SCIENCE WORKING PAPER 144

February

202

Using Theory, Markets and Experimental Methods to Improve a Complex Administrative

Decision Process: School Transportation for Disadvantaged Students

Charles R. Plott

(

California Institute of Technology

)

Gary Stoneham

(

Centre for Market Design, University of

Melbourne

)

Hsing

Yang Lee (California Institute of Technology)

Travis Maron (California Institute of Technology)

Abstract

he paper

studie

structural inefficiencies

of administrative decisions

demonstrated

in an

example

from

the

growing government

service

sector

The provision of school transportation to

disadvantaged children

touches

social concern

s and

regulations

related to

classical conditions

of limited competition.

The

decision

rocess is partitioned into key economic functions

the challenges

faced by the government

when dealing with environments in which theory

predicts market failures.

heor

ies

drawn from publ

ic choice

theory

and

theories of

auctions

suggest

policy changes

that were

experimentally tested

in the challenging environment

. Field

data demonstrate the new processes produced

improved

allocations. Subsequent laboratory

tests demonstrate the success can be attributed to the

principles used in the

theory

and

demonstrate a robust link connecting

efficient

behaviors observed in laboratory experiments

and behaviors found in field events

1

The financial support of The Rising Tide Foundation and The John Templeton Foundation are gratefully

acknowledged. The tireless efforts of governmental and educational personnel who enthusiastically supplied their

vast knowledge of institutional details o

f the problems faced by children, families, and institutions dedicated to

improving the lives of disadvantaged students must be acknowledged and admired. Special acknowledgement is

extended to Han Seo for his contribution to the route logistics model. The

technical support of the Caltech

Laboratory for Experimental Economics and Political Science made the project possible as was the assistance of

Jacqueline Lodman. The auction implementation team consisting of David Brooks, Veronika Nemes and Amy

Corman fr

om the University of Melbourne are acknowledged for their assistance in preparing for and hosting the

auction event.

1. Introduction

paper

explores

expands

and applies

a methodology for

implementing

market

based

replacements of administrative processes used

for

the

provision of governmental services

overnmental service

provision

one of the largest and fastest growing sectors of

governments

and

it is also

one of the most challenging

The a

llocation issues

often

span

multiple

economic

environments

that

heory suggest

are

incompatible

with

decentralized,

market

based

llocation

processes

With neither

theoretical

guidance

nor

examples of

market

successes

policy make

implement

administrative processes

onsensus often holds that

administrative processes are

inefficient,

but

no one seems to know how

fix

the

problems

This paper explores the issue with a focus on a specific case and asks if and how it can be

done

paper reflects a non

traditional approach

in which

the study of

a complex

administrative

process

reveals

that

there is no

implementable

, encompassing,

decentralized

, efficient

alternative.

The existing

administrative

process

examined in detail and

decomposed into

components

where key economically relevant decisions take place. The components

are then

examined

for features that might support

separate

market

based processes

The

examination

is structured around

natural

questions

: what are the inefficiencies or problems that

emerge

from the exiting administrative procedure; w

hich components of the problem

markets be

crafted to solve

and which cannot; and how can components be recombined into an allocation

process

Each

component

confronts

an economic environment

traditional

theories hold will

lead to

market failure

The

environments

include public goo

with the associated possibility of

free riding

, non

convexities

with the associated possibility of non

existence

, multiple equilibria

with

coordination

challenges

and thin markets

with the associated possibility of collusion

The

paper

focuse

on each of the

components

create

conditions that

might

support

market

based

process

designed to

avoid

the failure

suggested by theory.

The new processes are

designed,

studied experimentally

and then

implemented in the field.

The data from the

field

implementation

are c

ompared to the results of the original

administrative process.

The implemented policies produce results superior to the results of the

original, administrative

process. The study then explores the reliability of the model as the data

collection is moved between laboratory and field conditions.

Measurement taken from

the

field are implemented under laboratory conditions and thus laboratory

experimental methods

combined with field techniques are used as tests of robustness of background models. The

principles

at the foundation of

the economic models

are supported in al

l tests

and are thus

supported by both sets of data

The possible

lack of correspondence between laboratory

results and field data are often found in

methodological

discussion

“external validity” and

“internal validity”

. C

ases in which the corresponde

nce might fail are documented by

Harrison

and List (2004)

. The robustness approach used here avoids

some of the

philosophical issues

While the focus is on one example,

getting disadvanta

ed children to school and back home

he example

stands as

an exemplar of a class of cases

in which key allocation decisions are

made through administrative processes

in a large institutional setting

The economic

and the

implementation

problems

are broad

challenges

market

based

policies

Thus,

detailed

study of the special case

is more than a simple example. It

demonstrates

possible

methodology and

strategy for determining how appropriate markets might be

designed

implemented an

evaluated

as administrative process

replacements

that help

the

governmental

services recipients

and governments

Final e

valuation and

ssessment

the overall policy

design and

implementation are organized

through three

experimental testbed

questions

developed by Plott (1994)

(i)

proof of principle

did the system do what it was designed to do

;

(ii)

design consistency

did it do it for understandable reasons

–

the principle

s used in the

desig

–

opposed to

a lucky accident

;

and

(iii)

robustness

is the

real

world evidence

“robust” in the sense that

the results produced by

the system are

consistent with slight variations in

field and experimental

environment

These

questions differ from typical analysis

related to theory testing

exercises

The first reflects

the goal orientation of the design and the second reflects the role of underlying science. The

third reflects the

generality of the underlying theory and the

ield environment in which neither

full control

nor

complete observations exist.

deals with

the

reliability and

robustness of

theory as

the application is changed

from

laboratory experimental

conditions to

naturally

occurring

applications

and

implementation

from one environment to another

where

possibly unobserved differences exist.

The issues are sometimes

discussed

as problems of

external and internal validity.

2. The

Background

Policy Challenge

While the study is an economic policy

levant

to real people with real problems the

perspective is on the

underlying

economic principles that drive the results.

The

public

service

be studied

is transport

ation

for children

with autism

to and from a specialized school

The

school

itself

is held in high regard

attracting students from a

large geographic region

. The

ationale

for a coordinated transport service for students to and from school is obvious but

there are many complexities that impede the efficiency and effectiveness of

such services.

onsensus

among

administrators,

e schools’

faculty and student’s

parents

hold that the

current system of transportation

is poorly optimized.

It is

fully

subsidized

but the service

provided

does not meet the “client focus”

criteria established as a core objective of th

sponsoring policy

The current

transportation

system

takes significant time out of the

children’s day

Each day

he children, who require supervision, spend up to

four

hours

a day

riding a

large

bus to

and from

school

(two hours each way)

, arriving lat

e at school and at home.

The

long bus ride tire

the

children,

and

they are required to catch a connecting service

that can

be a source of additional frustrations. The cost is

time that could otherwise be devoted to

classes and family.

It is no surprise that p

arents often claim that their most difficult

and

challenging

task is getting the children to school and back.

Many are single parents with

multiple children and

constrain

ing

job schedules.

A comment often heard is: “Nothing goes

right”.

A wide consensus holds that a transportation problem

exists,

but improvements are

difficult.

2

The subsidy is through the Australian National Disabilities Insurance Scheme (NDIS). Travel services are funded on

an in

kind basis under

the NDIS but are procured through the Students with Disabilities Transport Program

managed by the Victorian Department of Education and Training (DET).

ransport services are provided b

y private operators through service contracts that are

allocated by the responsible government agency. A standard government

procurement

framework including pre

qualification, sealed

bid tender

followed by

bilateral negotiation

used to allocate and pric

e transport service contracts. Th

standard

process

essentially a

“beauty contest”

resulting in open contract terms

. B

ecause the

information is

asymmetr

ic, t

bus company has information about costs

possible route

and times that are not available to

the school procurement process

. B

ecause the

re is no obvious allocation mechanism,

negotiation is time

consuming. This allocation process relies on

inning firm

design the

transport service offered to students

and

is,

effectively, an administrative and informal way of

addressing the network externalities and other transaction complexities that c

annot

not be

resolved through standardized public sector procurement process.

A key insight is that the d

elegation of rou

te design and vehicle specification decisions to bus

operators has resulted in the emergence of transport services that reflect

the

profit incentives

implicit in the underlying scale economies

at the expense of

service quality

Thus, a form of a

principle

agent issue seems to

have

emerge

d in which the

decision

making

agents

’

preferences

(the bus companies)

are not closely aligned with those of the principle

and the principle has

limited information

Superficially, p

roperly specified

regulations with clearly specified quality constraints might

appear to be a solution

. However,

asymmetric information between the service provider and

the policy administrators creates difficulties.

An information gap also exists between the

preferences

of the parents, who are the intended beneficiaries of the service and the

regulators. Even if t

e regulators

are endowed

with information about

the

preferences that

might be implemente

, the regulators

o not have

adequate

information

due to

limitations

mposed by costs or the realities of network complexity. On the other hand, those that know

costs

and can influence the costs do not know the preferences or have limited incentives to

implement them

if they did know them

. On the surface it might appear to b

e an unresponsive

bureaucratic process but at the base is an asymmetric information an

institutional design

problem

Project

Overview,

Design

Objectives

and

Development

The project

methodology

was motivated by a design and experimental testbed approach to

policy

The overall goal was to

improve

the

service without significantly increasing cos

. Service

improvement was interpreted as implementing a “customer” orientation

where the customers

are

the parents and the school

Different dimensions of service were considered as influenced

by different governments responsible for the use of public funds

or public

safety, especially

where children and educational institutions are involved

. The overlapp

ing responsibilities and

authorities

created organizational conflict that

necessitated the support of upper levels of

government. This

key

support

of upper levels of government

provided access to officials and an

understanding that procedures and policies

could be modified if needed.

It is important to

recognize that we found no “governmental utility function” that might be optimized.

Although broadly understood as a transport problem, busses (in contrast with trains or trams)

are

relatively unconstrained form of transport network. The advantage of busses is

flexibility

because routes

can be assembled from segments of the existing roa

d network

, rather than

fixed rail infrastructure. However, other considerations

including: the

number and location of

students

;

safety

and supervision requirements;

budget

limits;

reasonable travel times,

and

other non

negotiable policy constraints

will in

fluence the bus network provided

conomic theory suggests why

this allocation

environment contains many features that can

cause market failure at both theoretical and practical levels. For example, the

pickup locations

have the properties of public goods, both

in regard to

location and

timing

of pickups

onflict

that

exist among the “customers”

creates

a “preference aggregation” problem with theoretical

and “impossibility” issues well known in the soci

al choice literature

The classical competitive

model for price determination is challenged by economies of scale related to vehicle capacities

and the zero marginal cost caused by excess vehicle capacity (partial fills)

due to problem

causing non

convexit

ies

Thus, equilibrium of classical models need not exist.

Other non

convexities

related to

individual route configuration,

multiple possible routes and the need for

coordination

, competition

with

multiple possible equilibria, challenge equilibrium existen

ce or

uniqueness.

Transportation cost reduction is

also

challenged by possible collusion among bus

operators organized

through a

bus company association.

These complexities suggest that t

here is no obvious

, implementable,

decentralized allocation

mechanism

that solves all of the

problem

A reductionist methodology was used to explore

and develop

possible

process

improvements

applied in sequence

. For this approach, the

problem was

deconstructed

into three components: servic

e quality preference

(discussed in

Section 4)

, route design

(Section 5) and

route allocation

(Section 6)

The

outcomes from th

alternative

allocation process

, behavior and dynamics are discussed in Section

Section

examines

the

success of the underlyi

ng science

by assessing

deeper and methodological

question posed in Section 1

It examines outcomes achieved in the field in terms of

proof of

concept

design consistency

and

robustness

We examine whether the

policy

mechanism

developed

perform

according to the basic economic principles

used to guide the design

. We

also

focus on the relationship between laboratory experiments and “naturally

occurring”

behaviors as encountered in field work

as an evaluation technique

A new technique is

developed in Section

and is applied to resolve the issue of robustness. Section

is a

summary of conclusions.

Service

uality

references

The first step

following

the establishment of

high

level

support

was to

formulate

and define

service

s to

be delivered.

Services took multiple forms with complex attributes

with different

implications for

market

based

provision

processes

Some quality attributes such as

vehicle

safety,

working with children accreditation and

supervision requirements

are

mandated by the

State

Supervision is provided by a

professional chaperone

whose task is to

attend to students’

needs

and maintain a safe and secure environment, particularly for younger students

and those

with severe disabili

. These supervision requirements can only be achieved with vehicles that

have a center

aisle

but this

eliminate

smaller vehicles

(

below 20 seats)

from consideration in

route design.

Other

ransport service

qualities

relating to maximum travel time, timely arrival at

school and student collection stations

were de

veloped from

discussions

among

school

administrators, teachers, and parents.

A maximum travel

time of no more than one hour

(each

way)

was

defined

for any stud

ent using the bus service.

Timely a

rrival

of the bus service

prior

to the start of the school day

was also identified as a required service

quality attribute

since

late arrival of the bus service imposes a negative externality

on all

students and teachers

attending the Northern School for Autism

A third service quality defined the

locations where

students would be

collected and returned each school day. Considerations such as: parking

space and access for parents,

safety,

shelter and proximity are taken into account

by school

administrators

The service qualit

ies

determination

is a social choice problem

choosin

the public goods (the

service qualit

ies

)

supply to

a population of parents and teachers

among

which service quality

preferences differed. The complex problem of determining public goods supply

was largely

facilitated

through the Public Choice mechanisms

multiple,

small group meetings and

consultations

with parents organized

and structured

by school administrators

The quality of

service with respect to

pick

up/drop

off locations,

safety, duration of journey, removing the

second leg and timely

arrival at school were the primary motivation to reform the previous

service

Cost was also an issue, but it was considered to be of second order.

Route

Design

Route design is the second component of the proposed allocation process.

Seventy

nine

of the

children attend

ing the senior campus of the Norther

School for Autism participate

. These

students attend school

two hundred days

a year

As noted

earlier, the previous allocation

process delegated route design to bus operators

The routes established under this system

had

children picked up at predesignated collecting areas and transported to a central hub

(the

junior campus)

where they

would then

change buses before finally being transported to

the

3

Details of the meetings are not documented. Impressions from discussion with the school administrato

r suggests

that no formal voting rule was in place, but one participant had veto power (the school administrator).

senior campus

The

second leg

of the bus journey

led to

the

late arrival

of travelling students at

he senior campus

(by around 30 minutes)

but also disrupted class for other students.

Each child was assigned to a pickup location

The p

arents had the responsibility of

transporting

their child to the pickup location at the scheduled

time,

so parents

had preferences about

locations

. This

assi

gnment

was a public good

decision

problem

the solution of which was

determined by

school/parent decision process

The determination of pickup locations

included issues of safety, parking and shelter

and

eventually resul

ted

in 35

pickup

locations,

with one to

seven

individuals picked up per location.

ata

from the previous service

and

experience with school transportation suggested that a stop to pick

up children required about

90 seconds per child at the stop. The logistics

challenge was to determine the number of buses

and the route of each

bus needed to transport all students to and from school given the service

quality attributes required. In this context, a route is defined by a sequence of pickup stations.

Service qualit

y requirements were implemented through:

)

prequalification (

e.g.,

safety,

supervision, seating configuration, accreditation of drivers and chaperones)

;

) pre

determined

(

e.g.,

pickup stations

and single leg service); and

iii

) constraints on the route f

ormation

methodology (e.g.

the one

hour maximum travel time, timely arrival and bus size

Factors

such as: traffic congestion, traffic light patterns, bus access to city streets and overhead

clearance

traffic management

arrangements

, on

bus disruptions

etc. had to be considered in

the route design process.

Technically, two large buses could be used to transport the 79 children

but that option was not

feasible

because of the

one

hour

maximum ride time, as the estimated time to stop for 40

children would be more than the maximum ride time.

The policy that the minimum size bus

would have a capacity of 20

(due to supervision requirements)

and

available supply of bus sizes

dictated that a

ll buses would be the same size. From an economic point of view, the

route

4

Exactly how this public choice process worked was not investigated. Available reports suggest that the school

administration met with sma

ll groups of parents and was vested with considerable authority of veto powers with

parents having direct input and voting. Accepted models and experiments suggest that the administrator’s most

preferred alternative would be in the core of the small group

decisions.

determination

objective became minimizing the number of buses used subject to the constraint

that all children were picked up and no child was on the bus for over an hour. Other cri

teria

such as minimizing the average time students were on the bus were considered

but

after

discussions with administrators

the other criteria

were not used. The distance between each of

the 1

190

pairs of pickup locations was determined

using street map

s and verified by Google

aps. Driving times between

the pairs of

pickup locations were based on traffic flow models

for the area and

sampled

though Google

aps.

data

from the 1190 pairs of pickup

locat

ons

ere

used to compute candidate routes

that would have all children picked up

with

90 seconds per child allowed for loading and unloading at a pickup location. The fact that

excess capacity would exist on the bus and the small number of children at each pickup location

(typically one to

three

with only six stops with four or more and one with seven) suggested an

efficiency

based assumption that no more than one bus would visit a drop

off

location

The collection of assumptions resulted in an estimate that seven buses would be used and thus

ven routes needed to be determined. A constrained optimization was performed to partition

pickup locations

to seven routes subject to the constraints that no route took longer than an

hour. The routes were not unique but were further refined such that th

e pick

up was consistent

with the direction the vehicle was traveling.

Even with constraints

hundreds of

thousands of

possibilities existed.

Google

aps

sampled at different times of day

was used to estimate the

time a route would take during the time that the bus would be operating. The routes were then

driven by members of the research team

check

route feasibility with respect to

traffic flow,

road conditions, obstructions

and other p

ractical issues. Those calculations resulted in seven

routes

, characterized in terms of

the locations o

pickups and the children to be picked up at

each

, that defined the

bus services to be procured.

The routes are illustrated in Figure

. The

economic

modeling work predicted that no child

would be on the bus more than an hour.

The previous routes

that the school had used in the

past

are illustrated in Figure

As previously mentioned, t

he former rou

tes were based on a

hub and spoke design that had students transported to a location where they changed buses.

The old transportation system

illustrates the

tradeoff between

reduced

cost

to the bus

company

and an

increased cost to families in terms of th

e time on the bus.

Once created by the research team, t

he routes were verified by the school and b

companies

that would sell the needed transportation services.

Once defined, e

ach route could not be

changed by the bus company hired to provide

services.

The decomposition of the

procurement

from a

single supplier of all routes to a competitive procurement of multiple routes

and

multiple suppliers

(discussed in Section 6)

was a major departure from historically used

procurement

procedures.

A dai

ly log maintained by the drivers

allow

for

a comparison between the model predictions

and the actual

travel

time.

The comparisons are contained in Section

One model error

was

due to a mistaken address.

measure

ment

error

was

due to a

congested intersection that was

subsequently avoided by the driver.

Figure 1: Analytically Determined Routes

Used in the

Auction

Figure 2: Previously Used (the old) Routes

Route

llocation

Auction Design

he seven routes

identified above,

were allocated to private

transport operators through

new

form of (

computerized

)

auction.

The auction was designed as a decreasing price, continuous,

simultaneous, multiple item auction.

The theoretical model for the auction is based on the best

response, competitive model.

Bidders select items and place bids in real time. Whether or not

the bi

d becomes a leader on the item or if a bid was replaced as leader was quickly apparent

during the auction. A countdown clock reset with any new leader in any market. The auction

ended and all markets closed if the countdown clock reached zero. The performa

nce success of

the continuous multiple item auction is well established in field applications for scale (Plott,

1997) and complex variations (Plott, Lee and Maron,

AER

2014). The basic theoretical

properties are analogous to multiple

items

, ascending price

auctions see

Demange, Gale, and

Sotomayor (1986) or for models based on discrete rounds see

Milgram (2000) or Plott and

Salmon (2004). For analysis of multiple unit auctions see Kwasnica and Sherstyuk (2013).

he school routes auction

is a new form of auction that

had important, untested features

Furthermore, the auction

was to be applied to environments that had never been studied.

Probity

restrictions

imposed

auction features that are new to the auction liter

ature. Thus

the

application r

equired a testbed phase.

The prominent and special features included:

(i)

Starting prices and decrements

The auction was a decreasing price auction, reflecting

the fact that the auction was a procurement as opposed to a

sale.

The decreasing price

feature implied the need for starting prices

(

the maximum price that a bidder could

offer on an item

)

and a decrement requirement

defining

the minimum difference

between a current leading bid and a new bid.

(ii)

Thin market

–

A combination of factors

including onerous prequalification, short

term

contracts

, a tight procurement schedule and opposition to a competitive allocation

5

Decreasing price auctions are typically considered as following the same principles (inverted) as increasing price

auctions. For comparisons in a clock auction context see Deck, Servatka and Tucker (2019).

6

The standard contract term is 10

years,

but a three

year contract was offered in this instance to synchronize with

other bus contracts.

process by the bus industry

led to only three bidders in the auction.

The thin market

feature r

aised an obvious possibility of collusion.

(iii)

Probity imposed substantive

interventions

Although th

auction was designed

specifically to address

complexities relevant to the allocation problem,

it deviated

significantly from the standard

procurement process

. This

created a range of procedural

and probity issues that had to be resolved.

The open auction format

(

needed to

accommodate common values for combinations of routes

)

was argued by probity to

contravene the standard government procure

ment practice in which

sealed bid is

lodged

by each bidder, is

confidential and cannot be revised. A considerable investment

was also made to justify and demonstrate the advantage of the proposed auction

format in which participants

can

revise their bids (down) in light of the

combination of

routes

in each bidders’ portfolio and the

bids placed by others.

A second probity

requirement

restricted bidders’ participation to only those routes identified in their

“

expression of interest

” used

as part of the

state’s

traditional procurement bidder

screening process

Two bidders expressed interest in all seven

routes,

but one bidder

identified

interest in only three

routes

Last minute c

hanges to auction software were

needed to implement this

requirement

so that

no bidder was aware of possible

constraints on any other bidder.

Other s

tandard procurement procedures were in

implemented and the probity official was assigned special powers in the auction.

6.1 Bids,

tarting

rices and

ecrements

The role of the auction was to allocate commercial contracts

for each defined route

to private

bus operators. Bids in the auction were

defined

in terms of the daily payment needed to collect

students from each pickup station, provide on

route sup

ervision

deliver students to school

and

return them at the end of each school day.

As indicated above,

three

year

contracts

were

offered

in which there are

200 school days per year.

The starting bid was

set

substantially

above previous contract

prices

and

prices for bus hire available

line

is information

provided

a crude and inaccurate guess about final auction prices

and thus how far prices would

fall during the auction.

A standardized decrement was set based

estimates about

the

number

of bids that would be tendered

guesses about bidding speed

, the financial impact

the decrement,

and

the impact of alternative decrement values on

how long the auction would

last. The resulting decrement chosen was

$25. As it turns out

all

these guesses were

substantially wrong

, which provides some insight about the lack of information

available at the

auction design stage of the project

. The final prices were lower than expected and the speed of

bidding was faster.

Starting prices were set

at AU$1550 per day per route and the decrement

requirement was

$25. Therefore, the starting value

of a contract

was

$930,00

(200 days

per year x

year

contracts

$1550

starting price)

and the decrement

value was

$15,000

(

$25 x 600

.2.

Testbeds and Auction Preparation

Prior to the auction

the timing of governmental decisions prevented any extensive use and

testing of the underlying theory.

That said, a long history of experimental auctions provided

important insights

and expectations re

garding auction performance and limitations

Tests were

primarily limited to software and instructions testing. However, the methods created for

testbed experiments proved useful for the more developed tests that will be addressed in

Section

s 7 and

ere the auction performance is discussed.

The multiple

items

, simultaneous

declining bids

auction

(as opposed to ascending) ha

not

been studied in the literature. New experimental procedures were needed to deal with the

reversed direction on the aucti

on. Subjects in testbeds were allowed to sell “services” with

value created by opportunity costs imposed in the form of clearly stated “outside offers”. A

bidder allowed to provide a service (

e.g.,

transportation on a route) could sell the service in the

uction or sell to the outside offer

provided by the experimenter

. When the price fell to near

the outside offer, a strategically behaving, optimizing agent would stop bidding to sell in the

auction and sell at the outside offer instead.

The price at which

the bidder stopped bidding is

called the “dropout price”.

The situation is analogous to the induced preference method

typically used in seller experiments, in which sellers in the auction must buy the item from the

experimenter at a

pre

determined cost befo

re agreeing to sell the item in an ascending price

auction with

competing

buyers

. The cost plays the role of minimum price at which the seller

would accept in an ascending price auction

analogous to how

opportunity provided by the

outside offer dictates t

he minimum price that the bidder would sell in the descending price

auction.

The difference is that in the typical seller experiments the seller is exposed to an

out

pocket

loss if the selling price is below the cost while in an auction with an outside

offer, the

opportunity cost, an

out

pocket

loss is not possible.

Other

government policy

issues created uncertainties

in the design and test

bed phase of the

auction

The s

pecial

probity

rules

noted above (Section 6 (iii))

had not been studied

and n

either

the exact number of bidders nor the exact rules

dictated by probity

were known until just a few

days before the auction. Probity issues

also

prevented the auction team discussions with

potentia

l bidders

so such sources of information were not available.

Thus, direct information

about possible cost or

expectations

were unavailable.

Testbed parameters were estimates of

bus company cost inferred from bus prices posted on the web together with the

estimated

times and distances in route models derived from theory. The

refore,

numbers could be off by

orders of magnitude.

In this design environment,

given the timing of probity decisions and the

official date of the auction set by the

regulators, only

o experimental sessions

consisting of

three experiments were

possible

and these

experimental sessions were dedicated to

the study

of the

two

bidding conditions

that might be

imposed

on the auction

by probity.

The probity

decision would be made at a date too late to do additional tests.

One

experimental

configuration restricted the bidder to winning no more than three routes and the second

configuration restricted the bidder to bidding on only three specific rou

tes.

Experimental parameters based on cost estimates were presented to subjects in the form of

opportunity costs

(See

Table

)

. The financial units are in terms of an artificial currency called

francs

worth $0.0015

/franc

Testbed

bidders were told that they had seven items to sell and

the money received was the subject’s to keep. The subjects could either sell at the auction with

winning bids or sell to a private offer

, where the private offer represented the subject’s

opportunity

cost in supplying the item. Private offers were

listed in

each subject’s

incentive

sheet

which was given to them before the start of the auction.

Table

. Testbed parameters: opportunity cost of four bidders

Item/ID

311

312

313

314

1167

1247

1327

1494

1332

1005

1085

1165

1324

1491

1164

1244

1013

1093

1260

933

912

992

1072

1239

1308

981

1061

1324

1109

1276

949

1029

Table

contains predictions conditioned on theory for both types of bidder constraints,

constraint 1 and constraint 2, considered by the government. Under

onstraint 1

bidder 311

could win no more than three items

and

under c

onstraint 2

bidder 311 could bid on

ly on items

3 and 6. Bidders could view only their own bids

the minimum required to place a bid

, and

the current leading bid (but not who placed it)

. Restrictions on bidders were unknown except to

the restricted bidder

and the number of other bidders was unknown to a bidder

Of the t

hree

testbed experiments

that

were conducted

, o

ne experiment implemented “constraint 1” and

two experiments implemented

“

constraint 2

”

Each bidder kn

ows

own cost with certainty.

When pla

cing a bid on a route the bidder knows

with certainty the bid

amount needed for it to

become a “leading bid” and th

a winning bid

should the auction end.

In the absence of additional information,

a natural model of behavior

s a

“best response”. Set

the bid equal to the existing (leading) bid minus the decrement

requirement

if the resulting bid is above

the bidder’s

cost

; previous bid minus bid equals

minimal

decrement

able

lists the theoretical prediction for

the price of

each route and the

experimental outcomes

for the testbed exercises

The theory holds that the bidder with lowest

cost wins and pays a price approximately equal to the cost of the second lowest cost bidder

minus the decrement requirement. In these

testbed

s the decrement was only 1f. Experience

suggests that experimental

subjects will not

trade for zero and

will

require a small profit in

order to trade

so for the testbeds the predicted change in price due to bids is not precise

The numbers

were

to be compared to theory

and were not considered as any form of

prediction about the upcoming

auction

given

that

the field parameters were unknown

The

pooled results of the

pilot

experiments are statistically indistinguishable from theory

suggest

ing

theory reliability.

While testbed results were sufficiently close to theory to

prevent

alarm about the

basic principles

, the parameters that might exist in the field were

still

totally

unknown

pportunity costs, bidding speed and variance could

be sources of

problems.

Furth

ermore, the possible behavior of the bidders due to the

business orientation

and

training

, the effects of monetary magnitudes and the propensity for collusion were also

unknown. The behavior of the experimental subjects seemed consistent with theory and

their

behavior indicated no recognition of the possibilities for collusion

by the bidders

In spite of

all

of the

unknowns, t

he key feature of dominant strategy seemed to work.

The

Auction

7

A regression of the experimental results on the theoretical predictions produces an intercept that is not

significantly different from zero and a

coefficient on the result of 0.96, standard error of 0.0074, t statistic of 13

and P

value of 6.32 E

, when expressed as an indicator of closeness of fit

Table

. Testbed Experiments: Theory and Experimental Results

experiment 20181125

Testbed 1

constraint 1

311 cannot win more than

three items

Testbed 2 and 3 constraint 2

311 can bid on only routes 2,3,6

item

theory

result1

item

theory

Result

1247

1228

1327

1325

1311

1085

1150

1085

1075

1100

1244

1229

1244

1175

1176

1013

1019

1260

1275

1222

922

956

1072

1050

1094

1061

1054

1061

1050

975

1029

1023

1029

950

1022

The auction was conducted

in the Experimental

conomics

aboratory at the University of

Melbourne

, Australia with

auction software

that

was housed in California on a Caltech

computer connected to the internet. Programs, internet speed, web interface and machine

compatibilities were

proofed and tested several days in advance. All computer interfaces were

inspected by the probity officer for policy compatibility. Only the probity officer and the Caltech

team viewed the auction and the real time flow of bids.

he bids submitted by

the

transportation

compan

ies

competing for routes were tendered by a

team of two or three company employees.

While the identity of participants was strictly

confidential before, during and after the auction,

nevertheless,

it is

possible

that bidders knew

their competitors through informal channels.

range of measures were taken to maintain

experimental standards

and bidder confidentiality. Separate information and practice sessions

were held for each bidding company

by a separate member of the auction team referenced as

an auction “coach”

;

on auction day,

participa

nts

arrived at separate, predesignated times and

were escorted to a room where they met their coach

;

bidd

ers were

allocated

separate,

private

bidding room

and

coach

es prevented i

ndividuals from using cell phones or wandering around

the facility, where they might see or learn the number of or learn the

identity of other teams

When the auction ended

separate administrators went to bidder rooms to verify the sale and

sign agreements. Bidders left

the auction

at separate times

with no information about prices

received by other bidders or other bidder identities. Bidders were asked about the number of

bidders and all suggested that the number was in the range of five to eight,

thus

the thin

market was not detectable.

Auctio

n Outcome

The final auction prices of routes and the auction winners are listed in Table

. Bus

transportation on all routes w

procured. One seller (ID 322) won five of the seven routes

(approx. 70%)

Another

seller (ID 321) won two of the routes

(approx. 30%)

and the constrained

seller (ID 323) participated aggressively but won none

(0%)

Prices were in terms of AU

payments per day for 200 days per year for three years.

Table

: Final Auction Prices (mea

n 6

84.8

)

Route

Auction final

(winning)

price

(

$/day)

winning Seller

Last dropout price

(second to last bid)

(

$/day)

322

656

322

695

322

750

322

741

321

625

321

725

322

840

Auction

ehavior and

ynamics

The auction lasted just under 15 minutes with 2

bids submitted, an average speed of one

bid

every

four

seconds. The final bid prices

, the dropout

bids for

all

routes and all sellers

, are in

Table

ropout prices

should not be confused with the prices a route commanded in the

auction. The dropout prices of a bidder are

the final bid

on routes

submitted by a bidder and

are thus

the lowest price the seller revealed as acceptable

, interpreted as a seller’s revealed

pportunity cost for the route

Approximations of

dropout prices

are

used in

the post auction

analysis of

Section 6

to calibrate the

pattern of implied equilibria.

Theory

used to fashion the

auction design p

redicts the

actual

seller

of transportation service in the route

will be the bidder

with the lowest

opportunity cost

and

the sale price will be the dropout value of the bidder that

has the next to lowest

opportunity cost

value

(minus the decrement requirement)

. Note that

the lowe

st acceptable value of the winning bidder will be below the winning bid. For the

winning bidder

the lowest

acceptable

value

not revealed.

Table

: Seller dropout values and seller ID

(

$/day)

Route

321

322

ID 323

Final (winning) prices

(mean 684, sd 73.8)

Winning seller ID: the

unknown minimum

value is below final bid

656

617

1335

617

322

695

670

745

670

322

900

725

750

725

322

741

716

1260

716

322

600

625

1310

600

321

651

725

800

651

321

840

815

1285

815

322

Figure 3 illustrates the time series of price decisions.

The dynamic structure of price formation

appears to be a series of price wars between a pair (the same pair, 321 and 322) of bidders that

continue bidding the price of a route down until one of the two drops out. During these wars,

bidder 323, the constrai

ned bidder, is the leading bidder on the three routes the bidder

was

able to (and constrained to) bid on. This bidder was unable to

participate in the “price wars”.

The waring pair, bidders 321 and 322, continue

focus on

a particular

route until one of

them

drops out and starts bidding on a different route. If the new bid is on one of the three items 323

can bid on

then 323 responds. Prices decrease until 321 or 322 shift attention to a different,

higher priced route where another price war begins

ric

es on th

three routes

323 can bid on

move at a similar rate of descent while prices remain high on other routes.

In brief, t

he high

priced routes attract bids from 321 and 322,

(possibly 323 if it is a route the bidder can bid on),

starting a new bidding

war

which continues until

all,

but one bidder

drops out

, then the

process

starts

again

on a new route.

Figure 3: Time Series of All Bids on All Routes by

idders

o evidence

exists

of collusion or

the

strategic bid reduction that might be expected in such

thin markets.

Experiments have shown that c

olluding bidders typically recognize the fragility of

collusive arrangements and do not return to the market after having stopped bidding in a

seeming

collu

sive agreement

(Li and Plott (2005) and Brown, Plott and Sullivan

(

2009)

)

Thereon, t

he pattern of repeatedly returning to a market to compete

in the routes

procurement auction

suggest

an absence of collusion. The time lag between a bid and a

responding bid (four seconds on average) is very short. The absence of any form of collusive

pattern suggests that the lack of information about the number of bidders, bidder identities or

bidding

patterns of any other bidder is a possibly effective design tool for thin markets. Bidders

in the route auction

have no information on which to condition collusion enforcing actions.

550

650

750

850

950

1050

1150

1250

1350

1450

1550

100

200

300

400

500

600

700

800

900

Price

Seconds since open

Rt1

Rt2

Rt3

Rt4

Rt5

Rt6

Rt7

321

322

323

Policy

Assessment

(

Proof of Concept

)

Did it

do what

it was supposed to do

The assessments turn on question

s posed in Section

The first

roof of

oncept

Did

uction

hat

upposed

There are essentially three

measures

used to assess the

results for

policy objectives.

The first is the time required in the auction compared to the

experiences with

the government’s

sealed bids

process

. The second is related to cost and

efficiency. The third is the quality of the services provided

as measure by

the time

that the

children are on the bus.

.1 Auction and Administrative Process Time

The difference in

execution (as opposed to set up)

time required for the conduct of the

previously used,

sealed bid

process

as compared to the

continuously

decreasing price auction is

dramatic. The route auction lasted just under 15 minutes with more than 200 bids submitted,

on average one every

four

seconds. By contrast, the

time consuming,

previously used sealed bid

proce

ss consisted of a

process of bid sub

missions followed by months of negotiations

after

winners were selected

he route detail

was

the substance of

subsequent negotiations between

the government and the auction winner

. The combination

these

processes

used in the past

were

reported to be

major

time

consuming

factor

that

were

avoided with the

continuous,

decreasing

bid

auction

used in the route

auction

. The improved speed of the route auction

could

be attributed to the crafting and auction of well defined, individual routes as

opposed to

a packaging of all routes

for sale

to a single seller

of transportation services

with major features

left to a negotiation process.

.2 Relative Cost: Auction Outcome and

Contracts

A comparison between the cost of transport services from the auction and a sample of existing

commercial contracts for similar transport services is possible. Eleven existing contracts were

sampled from information available on

a g

overnment website. The au

ction outcomes were

8

The times devoted to background activities times required for the auction are not included in

the measures, i.e.,

meeting times for obtaining agreement on services, software development, route determination, testing, etc.

Similarly, time required by the background activities devoted to the administrative process are not included.

then standardi

ed to account for the duration of the different contracts

four years,

two

hundred days

per year for

the existing commercial contracts vs

three years

, 200 days per year

for

the route auction

contracts

The auction a

chieved at least equivalent contract rates compared with the sample of contracts

allocated by the previous procurement method.

As mentioned previously, Table

contains the

cost for each of the routes procured by the auction. Table

contains a sample of eleven

contracts drawn from the governmental website and standardized for comparison with the

route auction.

rom Table

and

it can be seen that the

average equivalent contract prices

achieved through the auction

$695,

is slightly lower (about 1%) compared with the

commercial contracts reported

$703

. There is no statistical difference between the sample

of commercial contracts and the prices of

the auction. However, as will be noted below

the

service quality outcomes (travel times for children) are better for the auction contracts. This is

an important observation because it suggests that the auction approach was able to produce a

better service

at no higher cost.

Table

. Sample of dollar value of daily rate of governmental school bus contracts posted on

the web transformed to match distance and terms of the seven routes auctioned: mean

$703, sd 50.8

$807

$733

$666

$653

$745

$723

$665

$648

$743

$689

$661

.3 Routes and

Bus Travel Times (

Children

’s

Time Riding the Bus

)

Data

on the bus

travel

times

are available from the first semester of school operation. Three

questions are posed as assessments of the route determination models and methodology.

(1)

Were the models of transportation time accurate

(2)

Were the

goals met?

(3)

What were the

impacts

on individual students?

As stated previously, t

outes were designed to deliver the children to school

with ride

time

limited to one hour. A separate bus operated on each of the seven routes

collecting children

from the designated pickup stations

. Th

e structure

of the routes

was based on a purely

theoretical model. After the auction

the winners put buses into operation

and supplied

performance data

Each bus maintained a log of travel times and stops. The data from those logs are the basis of

the e

valuation reported here. Table

contains data for each of the seven routes collected from

the records of the bus drivers. The table also contains the predictions of the model used to

determine the routes and driving times.

Table

Route times in minutes: model and

actual for AM and P

Route

Route time in minutes (one way) for average

transportation to the school, return and total Daily

(minutes)

model

minutes

ctual

actual

Daily

actual

Daily error

of model

Fawkner

117

Heidelberg H

114

0.00

Whittlesea

127

Brunswick

Northcote

117

Eltham

106

Richmond

121

totals

384

419

380

799

Note:

error in route four reflects an incorrect address used in the model

The goal was to limit the time on the bus to one hour

both to and from school

. The model

predicted

that all routes would meet th

condition. The model predictions for

one

way

travel

are in the first column and as shown in Table

the predicted times are close to the constraint

(three or less minutes on four of the seven routes). The actual

travel

times

(both AM and PM)

are in

able

In terms of performance

the goal of no more than one hour on the bus was met.