Download The Role of Transparency in Procurement: Revealed vs. Concealed Scoring Rules in Sealed-bid A+B Auctions and more Lecture notes Operational Research in PDF only on Docsity!
The Role of Transparency in Procurement:
Revealed vs. Concealed Scoring Rules in Sealed-bid A+B Auctions
Bernardo F. Quiroga
School of Management, Pontificia Universidad Católica de Chile, bfquirog@uc.cl
Brent B. Moritz
Mary Jean and Frank P. Smeal College of Business, The Pennsylvania State University,
bmoritz@psu.edu
V. Daniel R. Guide, Jr.
Mary Jean and Frank P. Smeal College of Business, The Pennsylvania State University,
drg16@psu.edu
We study A+B procurement auctions. Here, a buyer requests offers that include price and non-price
attributes called “quality,” and bidders respond with bids including their price and the level of quality
provided. Using a laboratory experiment, we analyze the outcome of two sealed-bid first score scenarios:
One where the scoring rule that weighs price and quality is explicitly communicated to bidders before
they submit their offers, and another where the rule is only known to the buyer and concealed to the
bidders. In addition, we compare outcomes where the scoring rule is made visible only after the offers
are submitted. Our results show substantial losses to the buyer as a direct effect of non-disclosure of the
scoring rule (“transparency loss”), whereas sellers see their profits increase.
JEL classification:
D44, H57, C91, D82, D47.
Keywords:
A+B auctions; transparency; procurement.
History:
First version, April 20
th
- This revision, December 3
rd
Special thanks to Guillermo Burr, David Escobar and Andrés Cisterna from the Chilean Procurement
Agency, ChileCompra, for their help and time and for providing the research question this work
addresses. The opinions expressed here are those of the authors and do not necessarily reflect the
positions of ChileCompra nor of its executives. The authors are particularly grateful of JOM’s co-
editor-in-chief Suzanne de Treville, departmental editor Mike Galbreth, and two anonymous referees,
for their relentless support and feedback in improving this paper. We also thank thoughtful comments
from Robert C. Anderson, Ed Coulson, Terry Harrison, Tony Kwasnica, Suresh Muthulingam, Liang
Xu, Shouqiang Wang, Daniel H. Wood, Dan Nielubowicz, Raicho Bojilov, Francisco Ruiz-Aliseda,
Mauricio Larraín, Consuelo Silva, Jorge Tarzijan, and seminar participants at PUC Chile,
Universidad de los Andes, ISCI/Universidad de Chile, the 2015 INFORMS annual meeting, the 2016
POMS conference, and the 2019 DSI conference. All experimental sessions were conducted at the
Laboratory for Economics, Management and Auctions (LEMA) of the Pennsylvania State University.
Financial support came from Penn State’s Smeal College of Business through their Small Grants
program and the Smeal Chair in Operations & Supply Chain Management. Quiroga acknowledges
support from the Fondecyt Initiation Grant #11191014 (“Bidding Behavior in Complex Auction
Environments”). All errors are our own.
- Introduction
This article investigates the impact of a buyer’s commitment to award a contract based
on an announced scoring rule in a sealed-bid multi-dimensional (A+B) procurement auction.
Here, the buyer sets up the rules for an auction to purchase goods or services. Each potential
supplier submits an offer, including both the monetary price (the A component of an A+B
auction) and a non-monetary attribute (the B component) valued by the buyer. The buyer
ranks the offers of all suppliers using a scoring rule, where the weight on ‘B’ reflects the
buyer’s valuation for each unit of the non-monetary attribute.
Using a laboratory experiment, we compare the differences in buyer’s surplus and
supplier profits when buyers expressly communicate the weight they place on a non-price (B)
attribute, versus the case when buyers conceal this information from potential suppliers.
When these suppliers place bids that include both price and a non-monetary attribute, we
analyze the effect of buyer transparency of the evaluation criterion on bidding decisions. Our
results suggest that the buyers are better off by transparently communicating this
information. Conversely, sellers capture more of the available profit under non-transparency
because bidders, in general, tend to bid less competitively.
1.1. Motivation
The question about the impact of using a clear and transparent scoring rule on auction
performance originated during conversations with members of the research division of
ChileCompra, the governmental agency in charge of procurement in Chile.
Created in 2003, ChileCompra is the central agency for governmental procurement (Gur
et al. 2013, Corvalán 2013, Reyes 2015). ChileCompra is supervised by the President, has a
publicly-appointed Director, and is part of the Chilean Ministry of Finance. It regulates all
public sector auctions and framework agreements, and over 850 governmental offices use
ChileCompra’s electronic procurement platform to purchase and contract goods and services
from over 100,000 pre-qualified suppliers and vendors. These transactions amount to
approximately 3.5% of Chile’s annual GDP. One of the explicit goals for ChileCompra was to
increase transparency in governmental acquisitions while generating savings and efficiency
due to supplier competition. When procuring goods and services on behalf of different public
agencies, ChileCompra uses several procurement formats; this includes multi-dimensional
score auctions that incorporate price and one or more non-monetary measures.
order to produce a ranking of bids received; a scoring rule can be implemented to weigh these
components.
Several different institutional buyers use contract assignment auctions similar to what
ChileCompra does. For example, state departments of transportation in the U.S. frequently
assign contracts using A+B auctions (Lewis and Bajari 2011, Snir and Gupta 2011, Bajari et
al. 2014, Gupta et al. 2015), which considered the (negative) time to complete the contracted
service as the “B” dimension. Government procurement auctions frequently use sealed-bids,
and government purchases account for ten percent of gross domestic product (McAfee and
McMillan 1987). Private firms also use sealed-bid multi-dimensional auctions, for example,
to select among insurance companies administering a corporate health insurance plan for
their employees, where insurers bid on multi-dimensional service provisions as well as money
transfers (Zheng 2000).
We define buyer transparency as the public disclosure of the scoring rule to potential
suppliers before they submit their bids. When a buyer requests proposals, the buyer
communicates whether the contract will be awarded based on a specific, publicly announced
scoring rule, or if the final award decision remains at the discretion of the buyer. Our work
specifically investigates the impact of buyer transparency in sealed-bid, first score, A+B
auctions: All bids are revealed simultaneously at the end of the auction (sealed-bid), and the
contract is awarded to the best bidder subject to the characteristics included in that best offer
(first score).
We compare sealed-bid auctions with scores that are transparent (TS) or non-transparent
(NTS). In TS, bidders know the exact evaluation rule used by the buyer to rank bids before
they submit them (Che 1993, extended by Asker and Cantillon 2008). In NTS, the evaluation
rule used by the buyer is concealed from the bidder, forcing the bidder to come up with a
subjective estimate of what the buyer is looking for.
Several authors (e.g., Klemperer 2002, Asker and Cantillon 2008, Yoganarasimhan 2016)
have used the name beauty-contest to refer to auction formats like NTS, because when bidders
submit their bids, they choose the levels of each attribute which maximize their expected
utility given their own guess/estimate on the buyer’s subjective preference for quality. If a
buyer is a governmental agency whose interest is also to maximize benefits at a society level
(making society’s preferences their own), in a sense, bidders are trying to estimate on those
social benefit terms at large. The name “beauty-contest” is inspired by a comment made by
Keynes (1936) regarding beauty pageants: Any judge whose performance is evaluated on the
quality of his/her own choice for the contest, chooses a winner not based on his or her own
beauty preferences, but on what his or her own pre-conceived perception of what the general
public considers the most desirable beauty features. Unlike beauty-contests in a game-
theoretical sense (i.e., a “majority rule,” where decision-makers try to guess as close as
possible to x% of the average decision maker’s guess; see Nagel 1995), for the auction format
described here, decisions of bidders do not have power to influence the value of the buyer’s
preference for quality. This makes the belief formation process for a supplier impossible to
determine without additional simplifying assumptions.
Quality is what we refer to as the “B” component of an A+B bid. In this paper, quality
consists of any non-price attribute present in a product offered by a supplier that affects the
utility of the buyer. We assume these attributes to be verifiable, measurable, and
contractible/enforceable by the buyer, and can be used to rank bids and assign contracts. To
understand the notion of quality that can be incorporated into an auction, we refer to the
taxonomy offered by Ketokivi and Schroeder (2004). They identify three types of measures
that can be used to quantify quality attributes: operationally-defined measures, perceptual
measures, and quasi-perceptual measures.
Operationally-defined measures are quality metrics that are objectively quantifiable. In
procurement auctions, common examples include the time savings to fulfill a contract
(compared to a maximal acceptable delivery time), the specific quantity of materials used,
the hours of operation of an office, or the number of employees assigned to staff a point of
service during peak time. These and other measures are objective, reliable, and verifiable by
both bidders and buyers, making their contractibility and enforceability possible. These are
the least problematic for the buyer and bidders to communicate and assess.
However, buyers often consider other attributes that generate positive utility that include
perceptual components in their measure. Perceptual and Quasi-perceptual measures include
any metrics that are subjectively quantified by the buyer. In general, the consideration of
quality attributes with perceptual components in governmental procurement auctions is
problematic due to the impression of impropriety when buyer transparency is an explicit goal.
While purely perceptual measures are based on impressions and experiences, quasi-
perceptual measures also often have an embedded and underlying operational component.
For example, agencies sourcing via ChileCompra might want to include a perceptual quality
attribute called “bidder experience,” where inclusion of such an attribute reduces scores for
inexperienced or non-incumbent competitors. These attributes are difficult to measure
had to submit offers “in the dark.” Moreover, objectively, there are no guarantees that the
evaluation of “just right” from one specific instance of this revealed preference for quality
would be exactly repeated, especially if a different decision-maker would evaluate this quality
dimension for the buyer in the future.
Prior research has used market design and game theory tools to analyze different
hiring/purchasing auctions, providing useful theoretical insights when bids are either fully-
transparent (e.g., Che 1993), open progressing (e.g., Chen-Ritzo et al. 2005), or single-
dimensional (e.g., Kostamis et al. 2009). However, for a large number of the auctions
conducted by ChileCompra (sealed-bid, first score, multi-dimensional), there is no pure-
strategies-equilibrium solution when the assignment rule is non-transparent and feedback
is restricted to bid levels only instead of actual scores. Figure 1 displays the difference
between the two scenarios.
Figure 1: A+B sealed-bid auctions under comparison.
It is important to distinguish between sealed-bid auctions (the focus of our paper) and
open progressing auctions (the focus of most other studies). Briefly, in open progressing
auctions, sellers compete to supply a good or service at a progressively better offer for the
buyer, which improves during the duration of the auction. Several software platforms (e.g.,
Ariba, FreeMarkets) were primarily designed to implement open progressing auctions, where
each bidder is aware of the best competing bid during the auction, and may choose to improve
their bid over time. At the end of the auction, the seller who has offered the best bid for the
buyer is awarded the contract and is expected to supply under the specified price and terms.
In contrast, in sealed-bid auctions, each bidder submits a unique final offer. All offers are
simultaneously revealed, and the contract is awarded to the bidder with the offer that yields
Scoring rule (S) for sealed-bids A+B auctions in price (p) and quality (q):
𝑆 𝑝, 𝑞 = 𝑎𝑞 − 𝑝
𝑎 ≡ quality weighing parameter, 𝑎 > 0
Transparent Scores auction (TS):
Exact value of 𝑎 known to buyer and bidders
Non-transparent Score auction
(NTS):
𝑎 known to buyer only; bidders only know
that 𝑎 > 0 , but not its exact value.
the best score for the buyer, again expected to supply under the specified price and terms.
Bidders do not know the offers made by any competitors, and are prevented from adjusting
their bid in response.
Chen-Ritzo et al. (2005) could derive optimal bidding strategies for open progressing NTS-
like environments, where the buyer’s preference for quality is implicitly revealed via the
relative ranking displayed during the auction. The open-progressing environment is the key
driver of their analytical results. They studied multi-dimensional bidding in open progressing
score auctions with a scoring rule that could be inferred from the feedback generated by open-
progressing bidding. In such an auction, bidders revise their offers based on their relative
rank feedback as the bidding takes place – they can observe where their quality (speed of
delivery) is too low, or their price is too high, compared to the current winning bid, and submit
a revised counter-offer. In practice, the feedback provided gives a full display of the buyer’s
preference
2
In contrast, for sealed-bid cases, the NTS optimal bid is undetermined without assuming
a prior (a belief structure) on the buyer’s concealed preferences. For instance, the theoretical
work by Kostamis et al. (2006, 2009), considered quality, not as a decision variable for
bidders, but as the buyer’s private information about a bidder’s desirability. Their model
enables a buyer to discriminate among bidders based on that private information. In our case,
each bidder can affect their own desirability to the buyer through their chosen quality level.
Sealed-bid,
1st-score
Sealed-bid,
2nd-score
Open
progressing
bids
Single dimensional
bids (quality
exogenously
determined by Buyer)
Multi-dimensional
quality + price
bids (fully
determined by
Supplier)
Chen-Ritzo et al. (2005) √ √
Engelbrecht-Wiggans et al. (2007) √ √ √
Haruvy and Katok (2013) √ √ √
Fugger et al. (2015) √ √
Bichler (2000) √ √
Our work √ √
Bidding mechanism Number of decisions per bidder
Figure 2: Experimental research on Multi-dimensional procurement auctions
2 They also compared their main multi-dimensional setting to the case where the quality attributes
were fixed and bidding was on price-only, for benchmarking purposes. The multi-dimensional
mechanism strongly outperformed the price-only case in terms of buyer’s surplus.
of receiving offers might be problematic
3
. Thus, it is not surprising that in procurement
auctions, the exact selection criterion appears to remain hidden in some scenarios
(Elmaghraby 2007; Yoganarasimhan 2016; Anderson et al. 2016). Given these incentives to
reduce buyer transparency, our objective is to compare the performance of the two systems.
Therefore, our research problem consists in evaluating which of the two environments
between TS and NTS is preferable to the buyer, ceteris paribus.
To isolate the effect of transparency loss, participants in our experiments (as in the
experimental studies referenced here) were provided with perfect information about their
private cost structures – what is known as the independent private values auctions paradigm.
In practice, this precludes the possibility of bidders falling prey to a “winner’s curse,” where
they might win an auction at a cost higher than the revenue obtained from fulfilling the
contract. Similarly, we assume perfect commitment: A bidder who wins an auction event will
automatically supply the same price and quality levels offered, and the buyer will pay that
amount for the good. In reality, risks of default and hold-up exist, and Bajari et al. (2014)
discuss such effects in governmental procurement. However, variations in the levels of those
aspects are not part of this paper. All results from our research are also free from explicit
corruption, deception, or dishonesty. Thus, our results can provide bounds to any observable
differences.
- Auction Environment
This research was developed to help answer the question of the cost for a buyer to be non-
transparent
4
. A field experiment directly manipulating the information available would have
been desirable from a realism perspective, but it is problematic to conduct any empirical
3 While not directly addressed in our study, there are two main reasons why buyer transparency in
public procurement, in absence of buyer corruption, might not be pursued: First, accurately specifying
all contract attribute metrics and their weights might be difficult before obtaining bids. Second, some
competition agencies have argued that transparency may facilitate collusion among bidders.
Specifically to the second point, publicly transparent rules could provide cartel participants
information to ensure the stability of non-competitive agreements (Anderson et al. 2016). Indeed,
mandated public disclosure of winning and losing bid characteristics, including prices, non-price
elements, and identity of the bidders, can be a powerful tool for facilitating cartel formation (Kovacic
et al. 2006, Marshall and Marx 2012). More broadly, cartel-like implicit collusion occurs when
competition is reduced (e.g., Fugger et al. 2015).
4 Full details of the general mathematical model are included in Appendix A. Every mathematical
assumption we use is tied to practice, and we draw parallels of the framework with the existing
literature as well. Please see Appendix A for full details.
study that manipulates the trust or transparency environment: At ChileCompra, they
maintain that full commitment to the announced rules, information disclosure, and
enforceability, are all desirable market traits not to be sacrificed. Given these constraints,
we decided to contrast both auctions through an incentive-compatible controlled laboratory
experiment. We experimentally compare the decisions of bidders in sealed-bid auctions with
multi-dimensional decisions with and without disclosure of the quality-weighted scoring rule
(see Figure 1).
Consider a buyer that seeks to procure an indivisible good (which could be a physical good
or a service) for which there are several (N) potential suppliers called bidders. Bidders know
the number of competitors they face, (N – 1), before making an offer or bid.
The good is characterized by its price p, and a quantifiable non-price attribute
5
q
(“quality”), assumed to be verifiable and contractually enforceable: buyer prefers a higher q
and a smaller p. The buyer’s surplus from this good is 𝑆(𝑝, 𝑞) = 𝑎𝑞 – 𝑝, where a is a positive
constant (see Figure 1). This surplus is the exact scoring rule used by the buyer to rank
different offers (bids) coming from bidders. Therefore, without loss of generality, we assume
the buyer prefers scores as high as possible.
In principle, it is not necessarily true that a linear rule like 𝑆(𝑝, 𝑞) is a correct translation
of the buyer’s preferences (see Kersten 2014). However, the linear rule is a reasonable
assumption in practice: For example, many of the California Department of Transportation
A+B auctions that consider price and completion time use a linear rule to weigh offered
completion times, with those weights calculated as measures of the daily social cost to the
public for the closure of the specific infrastructure (a bridge or street) in question (Lewis and
Bajari 2011).
A supplier who wins a bid to supply a good of quality q at a price p makes a profit of the
ask price p minus the production cost C. The production cost C depends on quality q, an
efficiency factor called z and a fixed cost F; thus, the cost is 𝐶 = 𝑞
ଶ
/2𝑧 + F.
6
The fixed cost F
5 Although there could be multiple non-price attributes (the model in Appendix A allows for many
attributes), in all our applications q is a scalar. With a single non-price attribute, the additional
complexity in the bidding decision makes it more difficult for a bidder when compared to traditional
sealed-bid price-only auctions. In any case, asking bidders to submit bids with more than one non-
price attribute would likely make the differences between transparency conditions even more salient.
6
This cost function specification yields an elasticity of supply for quality constant and equal to one,
yielding a linear marginal cost for quality. This assumption is not unreasonable in practice: In their
There is some evidence in prior literature on beauty-contest auctions offered (cf. Binmore
and Klemperer 2002; Klemperer 2002) in the analysis of auctions used to assign 3G telecom
licenses in Britain and Europe. Most of their arguments against beauty-contests suggest that
auctions similar to NTS are time consuming and opaque, leading to political and legal
controversy and the perception of favoritism and corruption. All arguments in favor of
beauty-contests seem to rely on “better control” of the auction design, which Binmore and
Klemperer (2002) dismissed considering that one of the key aims of the auction was to “realize
the full economic value” for the state in the auction as long as other desirable objectives like
efficiency in assignment and promotion of competition were achieved. Since the TS auction
achieves those aims (see Asker and Cantillon 2008), and it is unclear if the NTS beauty
contest auction does so, this gives TS a partial edge over NTS.
In contrast, uncertainty aversion (Schmeidler 1989, Epstein 1999) would suggest in this
context that bidders in NTS would be more prone to choose “safer” bids (i.e., bids which
provide minimal profit for the winning bidder but a higher chance to win) over “less safe”
bids (i.e., bids which would provide larger profits for the winning bidder but which would be
less likely to win) due to uncertainty in the scoring rule. As bidders have no uncertainty over
the production cost in any scenario, uncertainty aversion would suggest that NTS would
increase the buyer’s surplus. In a TS auction, bidders do not face the “uncertainty risk” of the
scoring rule, predicting less aggressive bidding in TS over NTS, yielding smaller buyer
surpluses under TS than under NTS.
With all the above discussion under consideration, and understanding the opposing
underlying forces, we propose the following:
H1 – For sealed-bid, first score, A+B auctions, the buyer’s surplus is higher under
transparent scores (TS) than under non-transparent scores (NTS).
- Study 1: Comparing TS vs. NTS
Experimental design. We designed study 1 to be able to compare NTS and TS. We
implemented two treatments, each representing one of the two auction scenarios. The
random production efficiency factors, varying for all participants in all periods, were drawn
once and kept constant for all treatments
8
. Half of the participants were randomly assigned
to each treatment, and we used a between-subjects design. Subjects were matched in different
parallel groups of N = 4 bidders for 40 rounds. Subjects did not know the identities of the
bidders against whom they were competing, and were rematched in every round to different
competitors in the room. Subjects were informed at the beginning of the session of this
periodic anonymous rematching. In each treatment there were 24 participants, all sitting
simultaneously in the same room, with each participant sitting at their own isolated
computer station. Random rematching is a commonly used technique in experimental
auctions research (e.g., Katok and Roth 2004, Ockenfels and Selten 2005, Wan et al. 2012,
Chen et al. 2013). Although repeated social interaction would require analyzing the auctions
as a dynamically repeated game, the anonymous rematching in every round in a relatively
large room makes a non-dynamic framework appropriate. Specifically, subjects had no way
of knowing with whom they were matched, thus making it unlikely they could consider the
prior behavior of specific competitors. Subjects were not allowed to participate in more than
one session or more than one treatment.
Experimental parameterization. As noted, submitting an optimal A+B bid is a difficult
task, even when the scoring rule is known. In sealed-bid environments, the simultaneity of
the bidding precludes feedback on learning both the relative preference for alternative non-
winning offers as well as the appropriateness of a bidder’s decision, making the bidding
process psychologically more complex than with open bids. Therefore, we restricted our
design to a single non-monetary dimension.
We parameterized the efficiency factor 𝑧
to be uniformly distributed between 10 and 90,
the baseline cost to 𝐹 = 500, and the quality weight to 𝑎 = 10. These parameters provided
bidders a simple numerical setting to calculate an optimal bid that still captured real-world
features of the problem.
For the TS auction, with this parameterization, the theoretically-predicted bid (in terms
of quality 𝑞
∗
and price 𝑝
∗
) reduces to the expression below (for full details on the derivation,
please see the model in Appendix A):
8
Although the efficiency factors were re-drawn after every period, every auction had the same N
efficiency draws within a given round.
provided subjects with a decision tool that calculated costs and earnings using the efficiency
type and their inputs of price and quality. Bidders were also precluded from submitting bids
that would generate a financial loss, as is common in independent private value auction
experimental designs (e.g., Ockenfels and Selten 2005).
After all four bidders in an auction submitted their offers, each saw a results screen
showing if they had won or not, their own offered price and quality, the auction’s winning
price and quality, their profits, and a table with their history of all past rounds. This would
signal the end of the present round and the collective progress to the next.
To avoid introducing noise in subsequent bids, bidders did not receive any information on
the other auctions in the room taking place at the same time. The only information displayed
every period was the actual bid placed by the winner in case the winner was not the same
subject. This feedback structure is similar to the information provided by Katuščák et al.
(2015), and to treatment NF in Ockenfels and Selten (2005). Subjects did not know the
identities of their competitors in any round, and were truthfully informed they were
rematched every round with different subjects.
Results. We compare the auction results
9
in TS vs. NTS. Since we kept the same
realizations of efficiencies 𝑧
fixed by auction (round), we average the outcomes of each of our
six observed auctions per round. This makes every auction round comparable to each other
parallel round. Those averages are reported in Table 1, along with pairwise statistical
comparisons each for buyer’s surplus and for winning supplier’s profits.
9
The unit of analysis in Tables 1 and 2 is the individual auction, not the individual bidder. Appendix
C provides details on individual bidding behavior, and Appendix D explores individual bidding
dynamics over time. None of those results were central to answering our research question, and didn’t
help in providing insights in that direction, but we have made them available for the interested reader.
Table 1: Study 1 – Comparison of Buyer Surplus and Winner Profit per treatment
Buyer surplus
(Winning score)
Bidder outcome
(Winner profit)
Predicted Theory (
TS ) 2326.48 775.
TS 1842.65 693.
NTS 1614.65 919.
Wilcoxon (Theory-TS ) 5.309 2.
p -value < .0001 0.
Wilcoxon (TS-NTS ) 3.844 -4.
p -value < .0001 <.
Note: Sample size: T = 40 rounds, each using the mean results over all contemporaneous parallel
auctions per treatment. Values reported are averages per auction of each performance metric.
The buyer’s surplus (winning score) in the TS auctions was significantly larger
(Wilcoxon
10
signed-rank test 𝑧
ௐ
= 3.844, 𝑝 = 0.01%) than in the NTS auctions. Therefore, we
found statistical support for H1. On the other hand, winning bidders made a substantially
higher profit in the NTS auction compared to the TS case (Wilcoxon signed-rank test 𝑧 ௐ
- Study 2: The Introduction of Ex-Post Transparency (EPT)
As mentioned when the NTS mechanism was introduced, in some instances, it is
impossible for a buyer to transparently communicate either the exact weight or the metric
for quality. The idea of “I’ll know quality when I see it” is the motivation for our second study.
We introduced a new experimental treatment, which is an NTS-like beauty-contest with ex-
post transparency (EPT). In this case, the weight for the quality dimension is unknown to
bidders before submitting their bids, but it is revealed to all after bidding took place.
10
The Wilcoxon signed-rank test assumes that observations are pairwise-independent. Though not
strictly independent (subjects were grouped by session), remember that subjects were randomly and
anonymously rematched each round. Frechette (2012) remarks that, although several ways exist to
address possible session effects in the existing experimental literature, all introduce additional
problems that do not necessarily address the aim of the research question. Our use of average outcomes
(winner profits, buyer surpluses) as unit of analysis is suggested by Moffatt (2016), and follows the
practice of prior literature (e.g., Ockenfels and Selten 2005, Schmidt et al. 2003, Katok and Roth 2004,
Chen et al. 2013).
Just as the only difference between TS and NTS in study 1 was the revelation or not of
the quality weights, in study 2 the difference between the three cases was that, in TS, subjects
were informed of the quality weight before submitting their bid, while in EPT they were
informed of the quality weight after submitting their bid, and in NTS they were not informed
of the quality weight at all. As such, NTS and TS were designed identically to the earlier
study except for the varying weights.
Results. Since we kept the same realizations of types fixed by auction (round), we
averaged the outcomes (buyer’s surpluses, winner profits) of all parallel auctions taking place
in every period, for each treatment, similar to what we did for Table 1 in study 1. This makes
every auction round fully comparable to the same round conducted in each treatment. Those
averages are reported in Table 2.
With three levels of transparency, study 2 suggests that the results for the EPT treatment
introduced in study 2 are generally in between the “pure” versions of NTS and TS. The buyer’s
surplus is statistically indistinguishable between TS and EPT, and both are marginally
better than NTS. Note that because of the added difficulty for participants in study 2 of
having changing weights, the results comparing TS and NTS to have more variability than
in study 1.
In terms of profits for the winner, NTS still yields the highest profits, TS the lowest
profits, and EPT sits in between the two. Overall, the conclusions are consistent with study
1, but with statistical significances washing away. This is unsurprising considering the
change in weights over time makes the optimal bid task more challenging for TS participants.
Table 2: Study 2 – Comparison of Buyer Surplus and Winner Profit per treatment
Buyer surplus
(Winning score)
Bidder outcome
(Winner profit)
Predicted Theory (TS ) 3313.10 1046.
TS 2778.27 866.
EPT 2778.47 907.
NTS 2690.88 975.
Wilcoxon (Theory-TS ) 4.167 3.
p -value < .001 <.
Wilcoxon (TS-EPT ) 0.860 -0.
p -value 0.390 0.
Wilcoxon (TS-NTS ) 1.626 -2.
p -value 0.104 0.
Wilcoxon (EPT - NTS ) 1.331 1.
p -value 0.183 0.
Note: Sample size: T = 40 rounds, each using the mean results over all contemporaneous parallel
auctions per treatment. Values reported are averages per auction of each performance metric.
As a concluding remark, it is important to remember that the statistically insignificant
difference in buyer’s surplus between TS and EPT should not be interpreted as a policy
suggestion that EPT and TS are indifferent to the buyer, as we warned in the experimental
design section for study 2. Our experimental results are obtained in absence of corruption of
any kind, whereas in practice, a policy similar to EPT would likely open a chance for ex-post
weight adjustments from a buyer to give an advantage to a favorite bidder.
- Conclusion
We compared the performance of two similar sealed-bid, first score, multi-attribute (A+B)
procurement auctions, one with transparent scoring rules, and another concealing said rules.
Based on our results in study 1, the effect of transparency loss is statistically significant and
substantial in magnitude, providing empirical supporting to the idea that buyer transparency
helps buyers achieve a higher surplus. In addition, study 2 gives evidence that the losses in
buyer’s surplus from non-transparency are reduced when the rule is truthfully revealed ex-
post, conditional on absence of corruption such as ex-post favoritism.
