[dev] [patch] sizelimit in turba sql driver.

[Chuck Hagenbuch]

Quoting Marko Djukic <marko at oblo.com>:

> to work out that i'm on page 3 of 8? i'd need to know that there are 8
> total pages, no?

Only if you can't get the total number of results quickly. For SQL, a
count(*) gives you that very efficiently. Dunno about LDAP - but why make
the SQL case slow?

> > if there are a lot of results, but you only need one - you'll never
> > fetch
> > the whole result set if you just fetch the first slice in the backend.
>
> sorry, i don't follow you here.

If you always fetch every single page, you essentially assume that the user
wants to view each one. If they only want one that's on the first page - or
even in the first half of the result set - then you've retrieved too much
information.

> it might put extra load, but on the other hand with backend-only work
> like using modifyLimitQuery you are definately putting extra load.

Eh? You're comparing apples and elephants... And I'm not sure you can
justify the second half of that statement, anyways.

> two problems that kept coming to mind:
> a) minor, but how would the prefs based driver do the searching?

Anyone using the prefs driver for so many addresses that leaving this one in
PHP is a performance problem is a tad crazy.

I'm not sure on LDAP. But again, why shouldn't we optomize the cases that we
can?

> as above - how do we know that there should be 8 pages of total results?
> but i could be wrong, would doing a count() based total search then a
> limit search be less resource-intensive than doing a total search then
> slicing the data with php?

Quite possibly. Don't forget the overhead of caching the data somewhere -
tossing stuff into the session isn't free, either.

> and the other problem i had before was that in certain searches on large
> a DB a sql limit query would die, whereas i could manage to do a no-limit
> search and work on the data in php. i suspect i had to tweak something
> with my mysql, but never managed to hunt it down.

That doesn't really make sense - you can return a huge amount of data, but
not a small amount?

-chuck

--
Charles Hagenbuch, <chuck at horde.org>
The alligators were there, too, in a bathtub inside the house.