Docsity
Docsity

Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity


Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan


Guidelines and tips
Guidelines and tips

Query Optimization and Evaluation in Database Systems, Study notes of Database Management Systems (DBMS)

The concepts of query optimization and evaluation in database systems. The lecture, given by dr. Ala al-zobaidie, discusses the purpose of query optimization, the steps of query evaluation, and various optimization techniques such as heuristic, graph-based, cost-based, and semantic optimization. The document also includes an example using a typical tool and covers distributed optimization.

Typology: Study notes

2010/2011

Uploaded on 09/08/2011

rossi46
rossi46 🇬🇧

4.5

(10)

313 documents

1 / 5

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
DBDI 30/05/2007
Lecture 15 1
DBDI / Lecture 15
Query Evaluation,
Processing & Optimisation
Dr. Ala Al-Zobaidie
The slides are based on the textbook Database Systems by Thomas Connolly & Carolyn Begg
30/05/2007 Lecture 15 2
Objectives
Purpose of query optimisation
The steps of query evaluation
Static versus dynamic query optimisation
Optimisation Techniques
Heuristic
–Graphs
Cost Optimisation
–Semantic
An example with a typical tool
Distributed optimisation
30/05/2007 Lecture 15 3
SQL – What, Not How
The idea of declarative languages
like SQL is that you simply say
WHAT you want
You don’t say how the work is to be
done
It simplifies the life of the RDMS user
enormously
The y simply don’t have to care
about low level details
Only about the business problem
they are trying to solve
And the RDMS is free to control system
performance
But the work has to be done
And done quickly and accurately
30/05/2007 Lecture 15 4
Relational Algebra
Remember that underlying the relational
model is the Relation Algebra RA
Set and sub-sets and intersections and so on
The RDMS takes the SQL and converts it
to the RA
But if you remember RA there were lots of
ways to write down the same thing
And (from the RDMS) lots of orders in which
those operations can take place
30/05/2007 Lecture 15 5
Example - Different Strategies
Find all Managers who work at a London
branch.
SELECT *
FROM Staff s, Branch b
WHERE
s.branchNo = b.branchNo AND
(s.position = ‘Manager’ AND
b.city = ‘London’);
30/05/2007 Lecture 15 6
Strategy One
Staff Branch
staff.branchNo =
branch.branchNo
staff.position =
‘manager’
branch.city =
‘London’
σ(position='Manager') (city='London')
(Staff Staff.branchNo = Branch.branch No Branch)
pf3
pf4
pf5

Partial preview of the text

Download Query Optimization and Evaluation in Database Systems and more Study notes Database Management Systems (DBMS) in PDF only on Docsity!

DBDI / Lecture 15

Query Evaluation,

Processing & Optimisation

Dr. Ala Al-Zobaidie

The slides are based on the textbook Database Systems by Thomas Connolly & Carolyn Begg 30/05/2007 Lecture 15 2

Objectives

• Purpose of query optimisation

• The steps of query evaluation

  • Static versus dynamic query optimisation

• Optimisation Techniques

  • Heuristic
  • Graphs
  • Cost Optimisation
  • Semantic

• An example with a typical tool

• Distributed optimisation

30/05/2007 Lecture 15 3

SQL – What, Not How

• The idea of declarative languages

like SQL is that you simply say

WHAT you want

  • You don’t say how the work is to be done
  • It simplifies the life of the RDMS user enormously - They simply don’t have to care about low level details - Only about the business problem they are trying to solve - And the RDMS is free to control system performance

• But the work has to be done

  • And done quickly and accurately

30/05/2007 Lecture 15 4

Relational Algebra

• Remember that underlying the relational

model is the Relation Algebra RA

– Set and sub-sets and intersections and so on

• The RDMS takes the SQL and converts it

to the RA

– But if you remember RA there were lots of

ways to write down the same thing

– And (from the RDMS) lots of orders in which

those operations can take place

Example - Different Strategies

Find all Managers who work at a London

branch.

SELECT *

FROM Staff s, Branch b

WHERE

s.branchNo = b.branchNo AND

(s.position = ‘Manager’ AND

b.city = ‘London’);

Strategy One

Staff Branch

staff.branchNo = branch.branchNo staff.position = ‘manager’branch.city = ‘London’

σ (position='Manager') ∧ (city='London')

(Staff Staff.branchNo = Branch.branchNo Branch)

30/05/2007 Lecture 15 7

Strategy Two

Staff

Branch

position= ‘manager’ city= ‘London’

where staff.branchNo = branch.branchNo

30/05/2007 Lecture 15 8

Example - Different Strategies

• In RA we could write :

(1) σ (position='Manager') ∧ (city='London') ∧

(Staff.branchNo=Branch.branchNo) (Staff X Branch)

(2) σ (position='Manager') ∧ (city='London') (

Staff Staff.branchNo=Branch.branchNo Branch)

(3) ( σ position='Manager'(Staff))

Staff.branchNo=Branch.branchNo

( σ city='London' (Branch))

I haven’t shown this one

30/05/2007 Lecture 15 9

Calculating the cost

• Assume:

  • 1000 tuples in Staff; 50 tuples in Branch;
  • 50 Managers; 5 London branches;
  • no indexes or sort keys, results of any intermediate operations stored on disk, ignore final write

• Strategy no 2

• Strategy no 3

30/05/2007 Lecture 15 10

Phases of Query Processing

There is a trade off between static vs dynamic queries

Decomposing the Query

• Analyze the query

– Is it syntactically and lexically correct

– Check it is referring to real objects (relations,

attributes)

– Verify Operations (e.g. branch.city >

12/02/2006 makes no sense)

• Transform into suitable internal

representation

– A Relational Algebra Tree

Example - R.A.T.

  • If you haven’t noticed this is just like strategy two – you aren’t paying attention

30/05/2007 Lecture 15 19

Example Use of Transformation Rules

For prospective renters of flats, find

properties that match requirements and

owned by CO93.

SELECT p.propertyNo, p.street FROM Client c, Viewing v, PropertyForRent p WHERE c.prefType = ‘Flat’ AND c.clientNo = v.clientNo AND v.propertyNo = p.propertyNo AND c.maxRent >= p.rent AND c.prefType = p.type AND p.ownerNo = ‘CO93’; 30/05/2007 Lecture 15 20

Example 21.3 Use of Transformation

Rules

30/05/2007 Lecture 15 21

Example 21.3 Use of Transformation

Rules

30/05/2007 Lecture 15 22

Example 21.3 Use of Transformation

Rules

Cost Estimation

  • Relies on DB statistics
    • Can fall out of date
    • Or add to cost of updates
  • Will use schema driven knowledge of physical layout - i.e. existence of indexes
  • Strategies
    • Linear Search (Unordered file, no index).
    • Binary Search (Ordered file, no index).
    • Equality on hash key.
    • (In)Equality condition on primary key.
    • Equality condition on clustering (secondary) index.
    • Equality condition on a non-clustering (secondary) index.
    • Inequality condition on a secondary B+-tree index.

Estimating Cardinality of Join

  • Cardinality of Cartesian product is: nTuples(R) * nTuples(S)
  • More difficult to estimate cardinality of any join as depends on distribution of values.
  • This is the worst case though, cannot be any greater than this value
  • Better outcomes
    • Block Nested Loop Join.
      • Join two relations a row at a time
    • Indexed Nested Loop Join.
      • Use the index to go directly to the value in the joined table
    • Sort-Merge Join
      • Sort your relations and go through them one by one
    • Hash Join
      • Relations are partitioned according to a hash of the join condition

30/05/2007 Lecture 15 25

Pipelining

• If at each stage of processing we create a

complete intermediate relation (& perhaps

written that intermediate to disc)

– Materialisation

• However – if memory resources allow –

we can pipe the results of one operation

straight into another without

materialisation

– For example applying a 2 nd^ WHERE condition

to each tuple red and already filtered from a

base relation

30/05/2007 Lecture 15 26

Let’s have a look at some query analysis

• We are going to look at an execution plan

in SQL server

– Using the Query Analyzer

– Oracle has very similar facilities

– However with the graphical nature of SQL –

Server, you can see what is going on

30/05/2007 Lecture 15 27 30/05/2007 Lecture 15 28

Query Optimisation In Distributed DBs

  • When we distribute the database many more problems are added to assess - Another level of slow access (network accesses are orders of magnitude slower than disc accesses) - Hopefully it is clear that this is often just a parameter to the basic approaches we have already explored - However sometimes we will want to export processing - Let the remote machine do the join, projection, restriction first before sending data over the wire - If there is replication – choosing the right partner - Coping with network failures (the reliability of the join)

30/05/2007 Lecture 15 29

Summary

• Purpose of query optimisation

• The steps of query evaluation

  • Static versus dynamic query optimisation

• Optimisation Techniques

  • Heuristic
  • Graphs
  • Cost Optimisation
  • Semantic

• An example with a typical tool

• Distributed optimisation