Commit Graph

93 Commits

Author SHA1 Message Date
Tim Abbott 7ddcbd0d3a analytics: Set delivery_email in hacky test user creation code. 2018-12-06 15:33:28 -08:00
Steve Howell c0fd8660d2 subject -> topic: Fix analytics test (minor). 2018-11-14 23:24:06 -08:00
Vishnu Ks 201b99a6f8 models: Add USER_REACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks d0b89cbb44 models: Add USER_DEACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks ce3fffdbb2 models: Add USER_ACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 2c8effe9fe models: Add USER_CREATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Nikhil Kumar Mishra 26decb4c48 stats: Add 1day_actives::day CountStat to analytics tables. 2018-05-20 10:56:16 -07:00
Aditya Bansal 5adf983c3c analytics: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
Rishi Gupta fbd8dde1f8 invitations: Add LoggingCountStat to keep track of sent invitations. 2017-12-06 20:35:50 -08:00
rht 01885cdedc analytics: Use Python 3 syntax for typing (final). 2017-11-22 12:16:59 -08:00
Tim Abbott a0cfe45150 analytics: Wrap some longer lines. 2017-11-17 13:19:48 -08:00
rht d1689b5884 analytics: Use python 3 syntax for typing. 2017-11-17 13:16:49 -08:00
Tim Abbott 2b43a0302a python: Sort imports in smaller apps. 2017-11-15 15:55:49 -08:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
rht 691598a88b py3: Remove "from six.moves import range".
This is no longer required, since in Python 3, this is what the range
built-in does.
2017-10-17 23:28:14 -07:00
Rishi Gupta c7bdabbda8 analytics: Disallow non-UTC fill times in process_count_stat.
No change in behavior, but we aren't supporting non-UTC times in analytics
as a whole any more, so might as well change this check as well.
2017-10-05 11:22:06 -07:00
Rishi Gupta 0f31cddf49 analytics: Add management command to clear single stat. 2017-10-05 11:22:06 -07:00
rht 4494112862 analytics: Remove absolute_import. 2017-09-27 20:20:07 -07:00
Tim Abbott e3e2d9093c analytics: Remove references to UserProfile.objects.get. 2017-08-15 12:48:33 -07:00
neiljp (Neil Pilgrim) b782db48e1 mypy: Remove superfluous older 'type: ignore' annotations. 2017-08-08 11:27:51 -07:00
Vishnu Ks e5c5960faa analytics: Remove unused get_user_profile_by_email import. 2017-07-14 13:35:43 -07:00
umkay d9b23b39d3 mypy: Fix strict-optional in analytics. 2017-05-26 15:39:39 -07:00
Aditya Bansal 3f0b22ce31 pep8: Add compliance with rule E261 to test_counts.py. 2017-05-07 23:21:50 -07:00
Rishi Gupta 61bf445da4 analytics: Restrict fill_to_time to hour boundaries in process_count_stat. 2017-04-28 16:15:07 -07:00
Rishi Gupta 5e49da9285 analytics: Only update daily stats on day boundaries.
Previously we would update FillState for daily stats on hourly boundaries as
well. This would create two extra queries on the FillState table every hour
(for each CountStat), which adds roughly 50ms of extra processing for each
CountStat each day, as well as two extra lines each hour in the analytics
log. This can be a minor annoyance when backfilling stats.
2017-04-18 11:02:51 -07:00
Rishi Gupta b335ad2794 models: Add MIN_INTERVAL_LENGTH to UserActivityInterval.
Was previously a floating magic number appearing in both
zerver/lib/actions.py and analytics/lib/counts.py.
2017-04-18 11:02:51 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
hackerkid 55c3d12078 Replace timezone.utc with timezone_utc. 2017-04-16 12:28:56 -07:00
Rishi Gupta 49bd330304 analytics: Add class DependentCountStat and stat realm_active_humans::day. 2017-04-14 11:41:07 -07:00
Rishi Gupta 62de1cf898 test_counts: Modernize TestProcessCountStat tests. 2017-04-14 11:41:07 -07:00
Rishi Gupta 1e8d2b984d counts.py: Rename DataCollector-level operations to be more generic.
We're about to use these for DependentCountStats that will run SQL queries
on the analytics tables instead of the zerver tables.
2017-04-14 11:41:07 -07:00
Rishi Gupta 6dff22cbaf counts.py: Change check for LoggingCountStat to use isinstance.
I think this is more pythonic?

We could also get rid of LoggingCountStats altogether, since it's now just a
special case of CountStat (is_logging == data_collector.pull_function is None).
But I think it's nice to keep the distinction since they behave so differently.
2017-04-14 11:41:07 -07:00
Rishi Gupta 118b44d4f0 counts.py: Change DataCollector to take a pull_function argument.
This will allow us to appropriately generalize CountStat to include
LoggingCountStat and CustomPullCountStat. It'll also make life easier when
we introduce DependentCountStat.
2017-04-14 11:41:07 -07:00
Rishi Gupta 6369d23633 counts.py: Rename ZerverCountQuery to DataCollector.
Not the final form of DataCollector, but the name change causes a big diff
so separating it out.
2017-04-14 11:41:07 -07:00
Rishi Gupta b3991e2557 counts.py: Move CountStat.group_by into ZerverCountQuery.
Part of a larger refactoring to reduce cyclic dependencies between CountStat
and DataCollector (coming soon).
2017-04-14 11:41:07 -07:00
Rishi Gupta 341e1b54fc counts.py: Remove zerver_table from ZerverCountQuery.
Was only needed for filter_args, which are now gone.
2017-04-14 11:41:07 -07:00
Rishi Gupta 661de6bf25 counts.py: Remove filter_args argument from CountStat definition.
It turned out to not be that useful once we added subgroup. The previous
design of the CountStat object also assumed more reuseability of the *_query
strings than what ended up happening.

The filter_args also had some carrying costs:

* It's hard to be confident that filter_args other than the ones explicitly
  in our tests would have had expected behavior.
* The filter_args/join_args system is the most complex part of the CountStat
  object, and makes understanding the *_query strings unnecessarily
  difficult for a new contributor.
2017-04-14 11:41:07 -07:00
Rishi Gupta 6bb97db136 analytics: Add active_users_audit:is_bot:day. 2017-04-14 11:41:07 -07:00
Rishi Gupta 3d514c3e8d analytics: Add a default for the value column in assertTableState.
A default value of 1 is reasonable in this framework, especially for testing
things like LoggingCountStats.
2017-04-14 11:41:07 -07:00
Rishi Gupta 2f74ccabf9 analytics: Add 15day_actives CountStat. 2017-04-14 11:41:07 -07:00
Rishi Gupta 9b661ca91f analytics: Replace CountStat.is_gauge with interval.
Groundwork for allowing stats like "Monthly Active Users".

CountStat.interval is no longer as clean a value as before, so removed it
from views.get_chart_data. It wasn't being used by the frontend anyway.

Removing interval from logger calls in counts.py is not a big loss since we
now include the frequency (which is typically also the interval) in
CountStat.property.
2017-04-14 11:41:07 -07:00
Rishi Gupta d6c5c672d3 analytics: Add minutes_active CountStat. 2017-04-14 11:41:07 -07:00
Rishi Gupta 30024d0a8f models: Remove Realm.domain. 2017-03-25 19:55:48 -07:00
hollywoodno dd067c761a analytics: Separate private messages from group private messages.
This makes it possible for our graphs to show the group private
message counts as separate from 1:1 private messages.

Fixes #4102.
2017-03-20 11:46:29 -07:00
Rishi Gupta ceac6d9c59 analytics: Remove stray comment from test_counts.py.
The "actual test that would be nice to do" was indeed done!
2017-03-17 21:58:51 -07:00
Rishi Gupta 7c6f0033ed analytics: Add test for do_drop_all_analytics_tables. 2017-03-14 16:59:54 -07:00
Rishi Gupta 35f854a2fd analytics: Add test for do_aggregate_to_summary_table. 2017-03-04 16:46:09 -08:00
Rishi Gupta 8feea6c598 analytics: Add LoggingCountStat for number of users. 2017-03-04 16:46:09 -08:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Rishi Gupta 8bea47d6b5 analytics: Do a stylistic cleanup of TestProcessCountStat. 2017-03-03 16:12:12 -08:00