BugTask:+index times out with many bug tasks/nominations (eg. Bug #1)

My bugtask:+index branch has landed, which ameliorates some of the scaling here.

It doesn't help with the sourcepackage bitsy-queries, nor the linked-branch ones yet. That still needs addressing.

And more importantly, the activity + messages are still also always fully loaded; thats the primary driver for slowness (many MB of data loaded from the DB, some of it redundant).

Also
258. 9452 4594ms SQL-launchpad-main-master SELECT ValidPersonCache.id FROM ValidPersonCache WHERE id IN (1314307, 1956586, 567406, 316293, 2649926, 1096588, 1721003, 1894863, 568116, 640221, 829806, 434348, 1415953, 630520, 1485490, 979434, 240197, 507363, 232999, 1222974, 655632, 12153, 3179075, 2462629, 257227, 2262042, 587485, 682678, 266667, 455167, 240006, 927643, 510260, 2786294, 522995, 1173151, 1264573, 561150, 965919, 284122, 1478031, 1592139, 447413, 880678, 611742, 1325544, 2563568, 25412, 265595, 422808, 244942, 1505795, 27865, 3102960, 472520, 1027135, 1126094, 1788905, 331581, 427317, 752834, 248859, 697549, 1375881, 242016, 11339, 1943013, 1530728, 2222850, 1024979, 15177, 2263465, 37366, 1729457, 1497687, 3088750, 267488, 658848, 255083, 1538097, 925180, 1214230, 628496, 784248, 751359, 397262, 2412183, 501342, 1825652, 2401438, 1499801, 1105567, 31696, 559632, 3040549, 2961030, 1087623, 991564, 3213822, 1932730, 2238649, 433998, 2565202, 321809, 1938331, 1072844, 285765, 2167280, 235960, 614624, 646930, 402, 643300, 1284857, 1293812, 2621157, 426047, 307269, 762809, 731496, 1769811, 1398970, 3045555, 783141, 18779, 662989, 282674, 1989085, 1637440, 1839319, 1036946, 626848, 337937, 516047, 1958475, 393162, 1119550, 1743684, 1879502, 1133159, 808505, 1954289, 364432, 570133, 1288322, 3204533, 2096930, 1063197, 1866740, 433313, 1509294, 1298273, 477891, 1105452, 19204, 380955, 1910315, 1817181, 238532, 2993490, 457012, 732502, 1863720, 249253, 528542, 420019, 29244, 652820, 2642127, 501703, 1184863, 321282, 1754300, 965278, 2282454, 1809793, 3029646, 22895, 14869, 1986152, 1516617, 733260, 2459473, 1041525, 284188, 445564, 614329, 575323, 2411365, 1074968, 290904, 755021, 939225, 650628, 245086, 949283, 1031894, 2628205, 2568901, 1110254, 2526240, 295559, 1528153, 319332, 553488, 704174, 811621, 367862, 697534, 7038, 431787, 1844190, 1398727, 27695, 1911914, 1158100, 434420, 268302, 960453, 1224826, 1825950, 884847, 88, 1012331, 1891814, 639107, 399254, 503615, 1723272, 1622915, 12106, 1128055, 7439, 11521, 3098597, 581457, 261232, 1234404, 340769, 1124358, 408182, 1522563, 2372147, 2143553, 433342, 263232, 279749, 532048, 1099209, 1391844, 3267738, 988323, 3069255, 637651, 1515479, 1762629, 251380, 1372717, 980613, 958887, 559794, 2192607, 2410830, 2013520, 1042784, 981946, 19628, 25683, 925791, 1089489, 2915801, 905657, 685755, 2210353, 1113284, 1141032, 2969598, 14696, 615631, 471710, 2171238, 1292261, 1607509, 702176, 1392724, 1290217, 2042507, 1709419, 282763, 1123751, 1481472, 1706256, 285219, 2933642, 2235197, 624930, 251687, 710050, 1892377, 646092, 2107555, 1676381, 618650, 1133265, 2039266, 648401, 1127111, 1808152, 680772, 861769, 21372, 455815, 2039108, 941592, 1176389, 444646, 2304061, 668399, 1718024, 1932669, 3372051, 755705, 691318, 1458118, 1728702, 1931878, 643372, 412969, 430318, 1512892, 289704, 502288, 489442, 1122882, 1467166, 306460, 1885550, 242300, 797088, 3004809, 1143935, 3137, 1046540, 1428244, 2137197, 21829, 1814500, 279486, 2849355, 1501890, 953205, 1345217, 1042281, 817065, 1538653, 2842326, 415333, 539076, 3162720, 247668, 1131456, 616134, 429930, 909948, 350547, 2938849, 18540, 763329, 548...

It looks like this is getCurrentSourceReleases.

OOPS-1875QS36 shows that we have many Person queries. These are from task permission lookups, and should be easily precacheable. The nomination/targetting permission check is also a few queries like that.

Cool; are you going to address those cases?

Fixed in stable r12431 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12431>.

Theres likely more to tune here, but the getCurrentReleases angle seems well fixed.

Timeout with the new deploy, may be hot / cold, may be python - will look at the oops in the morning. Reloading it works now (it didn't before) so things are improved.