复杂查询下的Android SQLite性能

9

假设我有这样一种查询

String sql = "SELECT s.team_id, s.team_name, s.gp, s.w, s.t, s.l, s.go, s.ga, s.score, s.p FROM "
           + "(SELECT team_id, team_name, SUM (gp) gp, SUM (w) w, SUM (t) t, SUM (l) l, SUM (GO) go, SUM (GA) ga, SUM (GO)- SUM (GA) score, SUM (2*w+t) p FROM "
           + "(SELECT t._id team_id, t.name team_name, COUNT(CASE WHEN score_home IS NOT NULL THEN 1 END) gp, COUNT (CASE WHEN score_home > score_away THEN 1 END) w,"
           + " COUNT (CASE WHEN score_home = score_away THEN 1 END) t, COUNT (CASE WHEN score_home < score_away THEN 1 END) l,"
           + " SUM (score_home) go, SUM (score_away) ga"
           + " FROM team_table t LEFT OUTER JOIN match_table m ON m.team_home = t._id"
           + " WHERE t.tournament_id = ? GROUP BY t._id, t.name"
           + " UNION ALL"
           + " SELECT t._id team_id, t.name team_name, COUNT(CASE WHEN score_away IS NOT NULL THEN 1 END) gp, COUNT (CASE WHEN score_home < score_away THEN 1 END) w,"
           + " COUNT (CASE WHEN score_home = score_away THEN 1 END) t, COUNT (CASE WHEN score_home > score_away THEN 1 END) l,"
           + " SUM (score_away) go, SUM (score_home) ga"
           + " FROM team_table t LEFT OUTER JOIN match_table m ON m.team_away = t._id"
           + " WHERE t.tournament_id = ? GROUP BY t._id, t.name)"
           + " GROUP BY team_id, team_name) s"
           + " ORDER BY s.p DESC, s.score DESC, s.go ASC";

然后像这样使用

Cursor cursor = database.rawQuery(sql, args);

cursor.moveToFirst();
while (!cursor.isAfterLast()) {
    TeamStats stat = new TeamStats();

    stat.setTeamId(cursor.getLong(0));
    stat.setTeamName(cursor.getString(1));
    stat.setGamesPlayed(cursor.getInt(2));
    stat.setWins(cursor.getInt(3));
    stat.setTies(cursor.getInt(4));
    stat.setLoses(cursor.getInt(5));
    stat.setGoalsOwn(cursor.getInt(6));
    stat.setGoalsAgaist(cursor.getInt(7));
    stat.setScore(cursor.getInt(8));
    stat.setPoints(cursor.getInt(9));

    stats.add(stat);
    cursor.moveToNext();
}
cursor.close();

所以它从许多表中选择值,执行一些操作等等。正如您所看到的,这个查询非常复杂(很难调试),而且性能似乎不如我预期的那么好。我的问题是:

  1. 我能否使用某种准备好的语句来提高性能?
  2. 使用更简单的查询并手动处理它们是否会更快速?

6
改进你的数据库模式,反规范化它,添加索引等以使查询更简单快速。EXPLAIN QUERY PLAN 是查看sqlite如何处理你的查询的好方法。 - zapl
1
首先,您应该考虑以某种可读的方式格式化查询。StackOverflow不应该是展示您可以创建混淆的SQL语句的地方。 - tkr
1
@tkr 首先,因为拒绝你的更改而进行投票反对是很不好的行为。其次,String sql 部分是必要的,因为接下来的代码需要使用它。这是我拒绝你的更改的主要原因。你不能只是删除所有周围的代码并期望我接受它。下次,请考虑你的行为并自重。 - user219882
这不会给我带来显著的性能提升。您可以回答我在主题中写的那两个问题吗?这是我最想知道的。 - user219882
选项2- 是的,手动处理简单查询肯定会比对数据库执行更复杂的查询要更快。 - Chandrashekhar
显示剩余5条评论
5个回答

6
如果我是你,我会将SQLite数据库复制到主机上,然后尝试在某个SQLite GUI中手动执行它,同时用实际变量值替换绑定变量(?)。对于Windows上的GUI,我真的很喜欢SQLite Expert Personal,而在Linux上,sqliteman非常好。
在调试SQL时(在命令行或GUI中),一定要通过运行EXPLAIN和/或EXPLAIN QUERY PLAN来分析SQL语句。注意表扫描。您应该尝试通过添加索引来消除昂贵的扫描。但不要索引所有内容-这可能会使情况变得更糟。 通常,使用复合(多列)索引可以获得巨大的性能提升。请注意,在任何给定的表上,SQLite在运行给定的SQL语句时只能使用一个索引-因此,请明智选择索引。(请参见查询计划中的基本说明。)
至于您对Java与SQLite中数据处理的担忧-我认为完全优化(具有适当的索引等)的针对关系数据的SQLite查询几乎总是比在Java中手动处理此数据更快。这在您的情况下必须特别正确-所有数据基本上都是关系型。
不过,有一个小提示:您使用Java的Android APK可能比SQLite默认情况下拥有更多的内存-您可能需要使用setMaxSqlCacheSize()(相当于PRAGMA cache_size)为您的数据库增加SQLite缓存大小。 Android的默认值为10(最大100),请尝试增加它并查看是否对查询产生任何影响。请注意,桌面SQLite此设置的默认值要高得多-2000。

2
首先,我对SQLite并不了解,但是我认为它的行为会比较像MS SQL-Server。
通常,类似这样的简单查询的性能问题通常与索引缺失有关,导致全表扫描而不是部分表扫描或表搜索。如果您没有在team_table.tournament_id上建立索引,则SQLite将不得不扫描整个表来执行“t.tournament_id =?”操作。对于match_table.team_home和match_table.team_away也将发生同样的情况:缺少索引将导致join操作时需要进行全表扫描。
对于其余部分,您可以通过两种方式简化您的查询。第一种是放弃外部子查询并在Order by中使用表达式或列排序;即,您可以用“ORDER BY SUM (2*w+t) DESC,SUM(GO)-SUM(GA) DESC,SUM(GO) ASC”替换“ORDERBY s.p DESC, s.score DESC, s.go ASC”,并且省略subquery s。
第二种方法是将UNION替换为单个查询,同时在m.team_home和m.team_away上执行左连接操作:
... FROM team_table t LEFT OUTER JOIN match_table m ON (m.team_home = t._id OR m.team_away = t._id) ...
之后,很容易根据t._id等于m.team_home或者m.team_away的情况正确计算各种分数。这样,不仅可以取消UNION,还可以取消第二个子查询。
最后,您必须检查使用Left Join的情况;因为我不确定是否真正需要使用常规的Inner Join。
之后,您将得到一个简单的联接查询,带有Group By和Order By,并且没有子查询或Union,并且可能根本没有Left Join。然而,在这一点上,Order By中的表达式可能已变得有点复杂,因此您必须决定保留它们,恢复一个子查询或使用列排序(我最喜欢的选择)。
删除Union后,查询的性能应该至少提高两倍,但是最终要获得良好的性能,最重要的要求将是具有所有适当的索引;否则,如果SQL服务器需要执行多个全表扫描,则性能永远不会好。

1

个人建议在Android上尽可能保持查询和数据库结构的简单,通过代码进行主要处理。

原因之一是,复杂的数据库结构与需要处理不同版本应用程序的升级和降级而不丢失数据的需求混合在一起,很快就会失控。我现在倾向于以一种NoSQL的方式设置和处理数据。

另一个原因是,SQLite缺少许多在实际任务中所需的功能,最终你还是会通过代码来处理数据。例如,没有三角函数,因此查找最接近的项目可能会变得复杂;)

private String getRelitiveDistanceQuery( double lng, double lat, int max){
    return "SELECT *, " +
    // NOTE: this long query was done because there are no trig functions in SQLite so this is an series expansion of some of the functions
    "((3.14159265358979/2-( ((("+Double.toString(lat)+"*0.0174532925199433)-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/6+("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/120-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/5040)*((`lat`*0.0174532925199433)-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/6+(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/120-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/5040)+(1-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/2+("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/24-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/720)*(1-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/2+(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/24-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/720)*(1-(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/2+(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/24-(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/720))+1/6*((("+Double.toString(lat)+"*0.0174532925199433)-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/6+("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/120-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/5040)*((`lat`*0.0174532925199433)-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/6+(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/120-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/5040)+(1-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/2+("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/24-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/720)*(1-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/2+(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/24-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/720)*(1-(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/2+(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/24-(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/720))*((("+Double.toString(lat)+"*0.0174532925199433)-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/6+("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/120-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/5040)*((`lat`*0.0174532925199433)-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/6+(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/120-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/5040)+(1-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/2+("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/24-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/720)*(1-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/2+(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/24-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/720)*(1-(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/2+(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/24-(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/720))*((("+Double.toString(lat)+"*0.0174532925199433)-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/6+("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/120-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/5040)*((`lat`*0.0174532925199433)-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/6+(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/120-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/5040)+(1-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/2+("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/24-("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)*("+Double.toString(lat)+"*0.0174532925199433)/720)*(1-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/2+(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/24-(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)*(`lat`*0.0174532925199433)/720)*(1-(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/2+(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/24-(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)*(("+Double.toString(lng)+" -`lng`)*0.0174532925199433)/720)) ))) AS relDistance " +
    "FROM `"+TABLE_ITEMS+"` ORDER BY relDistance ASC LIMIT "+Integer.toString(max);
}   

我编写了一个Perl脚本来生成这段代码,它可以扩展三角函数,并且实际上效果还不错,但是它难以管理,我不建议使用。

0

如果您正在使用预处理语句,则对您有益,因为: 1. 预处理语句更加安全 2. SQL 注入更加困难 3. 它们并不是特别复杂 4. 维护很容易


我知道这些,但我想知道在Android中是否有一个返回Cursor的_PreparedStatement_。 - user219882

0

虽然不是关于快速查询的答案,但是:您可以尝试使用额外的辅助表,并通过在实际数据表上定义触发器来填充它们。这样,您将拥有大部分聚合数据准备就绪,并且查询会更简单。


网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接