Commit Graph

877 Commits

Author SHA1 Message Date
Zixuan James Li e836aa31ad test_support_views: Extract lear_realm.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 15:03:26 -07:00
Zixuan James Li aa5d540845 test_support_views: Add create_invitation helper.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 15:03:26 -07:00
Zixuan James Li c727f1f65d test_support_views: Add get_check_query_result helper.
This is a prep commit for a refactoring that fixes an issue with plan
data not being displayed when the realm is displayed by the query result
of users or confirmation objects.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 15:03:26 -07:00
Zixuan James Li 0d93257111 support: Extract PlanData as a dataclass.
This avoids monkey-patching `CustomerPlan` and other related information
onto the `Realm` object by having a separate dictionary with the realm
id as the key, each corresponds to a `PlandData` dataclass.

This is a part of the django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 15:03:26 -07:00
Zixuan James Li eeaeb5a821 confirmation: Refactor get_confirmations to accept confirmation ids.
Two of the callers of `get_confirmations` uses a `QuerySet` of confirmation
objects instead of their ids to filter the confirmations. This refactors
`get_confirmations` so that it is typed to accept `Iterable[int]` that
is a list of ids.

It's worth noting that this might be less performant than the previous
approach since it requires more queries when we force the ids into lists
without having django creating a nested query. But the performance
is not a concern here compared to clarity.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 14:46:51 -07:00
Zixuan James Li 0528935b82 stats: Tighten function signatures with generic QuerySet.
Luckily `QuerySet` supports type variables. This allows us
to type table_filtered_to_id more accurately.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-07 11:28:06 -07:00
Zixuan James Li 81ab0b3615 user_activity: Tighten function signatures with generic QuerySet.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-07 11:27:43 -07:00
Zixuan James Li 32ad110af2 analytics: Tighten function signatures with generic QuerySet.
Note that the `list` conversion before assignment to `all_records`
is not necessary for its usage in `realm_user_summary_table` from
a typing perspective.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-07 11:27:43 -07:00
Zixuan James Li ab1bbdda65 typing: Broaden type annotations for QuerySet compatibility.
To explain the rationale of this change, for example, there is
`get_user_activity_summary` which accepts either a `Collection[UserActivity]`,
where `QuerySet[T]` is not strictly `Sequence[T]` because its slicing behavior
is different from the `Protocol`, making `Collection` necessary.

Similarily, we should have `Iterable[T]` instead of `List[T]` so that
`QuerySet[T]` will also be an acceptable subtype, or `Sequence[T]` when we
also expect it to be indexed.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-07 11:27:42 -07:00
Zixuan James Li fd9a0f4274 typing: Apply trivial none-checks with assertions as necessary.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li 426f8ce385 tests: Replace `HttpResponse` with `TestHttpResponse.`
Since `HttpResponse` is an inaccurate representation of the
monkey-patched response object returned by the Django test client, we
replace it with `_MonkeyPatchedWSGIResponse` as `TestHttpResponse`.

This replaces `HttpResponse` in zerver/tests, analytics/tests, coporate/tests,
zerver/lib/test_classes.py, and zerver/lib/test_helpers.py with
`TestHttpResponse`. Several files in zerver/tests are excluded
from this substitution.

This commit is auto-generated by a script, with manual adjustments on certain
files squashed into it.

This is a part of the django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-08 11:25:03 -07:00
Zixuan James Li a142fbff85 tests: Refactor away result.json() calls with helpers.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-06 23:06:00 -07:00
Anders Kaseorg dede49ad78 test_counts: Rename confusing assertCountEquals helper.
It looks too much like the standard TestCase.assertCountEqual method.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-03 12:24:57 -07:00
Zixuan James Li d5517932cd typing: Use BaseDatabaseSchemaEditor in place of DatabaseSchemaEditor.
This is a part of #18777.

Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-30 14:18:53 -07:00
Anders Kaseorg fd16f97d6b python: Excise None from pointlessly nullable booleans.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-27 12:40:14 -07:00
Sahil Batra 61365fbe21 invites: Use expiration time in minutes instead of days.
This commit changes the invite API to accept invitation
expiration time in minutes since we are going to add a
custom option in further commits which would allow a user
to set expiration time in minutes, hours and weeks as well.
2022-04-20 13:31:37 -07:00
Anders Kaseorg cc30ed8ec7 actions: Delete zerver.lib.actions.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:38 -07:00
Anders Kaseorg e01faebd7e actions: Split out zerver.actions.create_realm.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:37 -07:00
Anders Kaseorg 59f6b090c7 actions: Split out zerver.actions.realm_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:37 -07:00
Anders Kaseorg eb4e9fe1e7 actions: Split out zerver.actions.message_flags.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:36 -07:00
Anders Kaseorg cbad5739ab actions: Split out zerver.actions.create_user.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:35 -07:00
Anders Kaseorg d7981dad62 actions: Split out zerver.actions.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:34 -07:00
Anders Kaseorg 6168c0110a actions: Split out zerver.actions.user_activity.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:32 -07:00
Anders Kaseorg ca8d374e21 actions: Split out zerver.actions.invites.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:31 -07:00
neiljp (Neil Pilgrim) c3317ebff8 stats: Adjust Website mapping to Web app for consistency. 2022-04-14 11:59:36 -07:00
neiljp (Neil Pilgrim) 08477ed5e9 stats: Add mapping for ZulipTerminal id to Terminal app. 2022-04-14 11:59:36 -07:00
Steve Howell 8f99894302 streams: Extract stream_color library.
This is a pure code move.
2022-03-14 18:01:36 -07:00
Steve Howell a90d9ef536 unread: Remove unused client parameter. 2022-03-10 10:04:51 -05:00
Sahil Batra 392b17da5f invite: Add backend support for "Never expires" option.
The database value for expiry_date is None for the invite
that will never expire and the clients send -1 as value
in the API similar to the message retention setting.

Also, when passing invite_expire_in_days as an argument
in various functions, invite_expire_in_days is passed as
-1 for "Never expires" option since invite_expire_in_days
is an optional argument in some functions and thus we cannot
pass "None" value.
2022-02-24 16:32:19 -08:00
Anders Kaseorg 21cd1c10b3 docs: Add missing space in “time zone”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-24 14:05:12 -08:00
Anders Kaseorg 1629d6bfb3 python: Reformat with Black 22 (stable).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-18 18:03:13 -08:00
Aman Agrawal 7614f2203a pricing: Replace "Zulip Standard" with "Zulip Cloud Standard".
Case sensitive replace.
2022-02-09 11:00:24 -08:00
Lauryn Menard 3be622ffa7 backend: Add request as parameter to json_success.
Adds request as a parameter to json_success as a refactor towards
making `ignored_parameters_unsupported` functionality available
for all API endpoints.

Also, removes any data parameters that are an empty dict or
a dict with the generic success response values.
2022-02-04 15:16:56 -08:00
Anders Kaseorg 78e54a0d7a python: Replace deprecated jinja2.utils.Markup with markupsafe.Markup.
Fixes “DeprecationWarning: 'jinja2.Markup' is deprecated and will be
removed in Jinja 3.1. Import 'markupsafe.Markup' instead.”

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:22:48 -08:00
Anders Kaseorg 1696144df7 docs: Consistently hyphenate “self-host” and “self-service”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-05 16:21:35 -08:00
Eeshan Garg 2cdaae681d actions: Rename do_change_plan_type -> do change_realm_plan_type.
We will soon be adding an equivalent function for RemoteZulipServer,
so it makes sense to rename this function to be more descriptive.
2021-12-06 16:18:53 -08:00
Eeshan Garg b325a4f1be realm: Rename plan type constants to be more descriptive.
It is confusing to have the plan type constants not be namespaced
by the thing they represent. We already have a namespacing
convention in place for constants, so we should use it for
Realm.plan_type as well.
2021-10-19 12:20:39 -07:00
Vishnu KS aff52722a7 billing: Add PLUS as a plan_type to Realm. 2021-10-15 17:27:50 -07:00
Eeshan Garg 1dec97c925 analytics: Use user IDs to get user activity summaries.
Using user IDs instead of emails is more reliable since users can
have arbitrarily complex emails that are hard to encode in a URL.
This has led to NoReverseMatch exceptions in the past.
2021-10-13 15:30:27 -07:00
Eeshan Garg f25230c7d4 analytics/support: Add ability to edit realm org type. 2021-10-07 14:08:11 -07:00
Mateusz Mandera 3205f680c1 do_invite_users: Turn some args into kwargs-only. 2021-09-10 16:53:03 -07:00
shanukun 8c1ea78d7d invite: Extend invite api for handling expiration duration.
This extends the invite api endpoints to handle an extra
argument, expiration duration, which states the number of
days before the invitation link expires.

For prereg users, expiration info is attached to event
object to pass it to invite queue processor in order to
create and send confirmation link.
In case of multiuse invites, confirmation links are
created directly inside do_create_multiuse_invite_link(),

For filtering valid user invites, expiration info stored in
Confirmation object is used, which is accessed by a prereg
user using reverse generic relations.

Fixes #16359.
2021-09-10 16:53:03 -07:00
PIG208 7386918539 typing: Use accurate type hints for dictionaries.
This fixes the mypy errors related to dictionaries with django-stubs.
2021-08-20 06:02:28 -07:00
PIG208 c99c423c81 support: Remove `is True` in comparisons.
This is a followup for #19432 that fixes a remaining issue of the
original PR.
2021-08-10 12:08:13 -07:00
Anders Kaseorg 09564e95ac mypy: Add types-psycopg2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
PIG208 94685e1afb analytics: Refactor the support view to use REQ. 2021-08-08 17:11:18 -07:00
Tim Abbott 65557c458d stats: Avoid referencing remote tables outside conditional. 2021-08-03 11:15:39 -07:00
PIG208 ec99c1b58d analytics: Correctly type aggregate_table with Union.
Since mypy doesn't accept redefinition of the same variable within the
same scope, we need to use type annotations with Union to correctly
type aggregate_table. Note that the type cast is necessary for mypy to
narrow the type of aggregate_table.
2021-07-27 12:17:01 -07:00
PIG208 6ab42b7663 analytics: Use type alias for `fixture_data`.
This saves us from repeating the lengthy type annotation for
`fixture_data`.
2021-07-27 12:17:01 -07:00
PIG208 e517f967b8 typing: Amend inaccurate type annotations.
This amend some type annotations that turn out to be inaccurate with
django-stubs.
2021-07-27 12:17:00 -07:00
PIG208 caaa424ef5 typing: Use assertions for broader types.
For types like `Union[Realm, UserProfile, Stream]` and
`Union[AnonymousUser, AbstractBaseUser]`, we need assertions to
tell mypy which type we would be expecting.
2021-07-27 11:44:54 -07:00
PIG208 8121d2d58d typing: Fix misuse of HttpResponse.
Amend usage of HttpResponse when appropriate.
2021-07-27 14:31:19 +08:00
PIG208 d2af20eb2a typing: Do explicit type conversion when appropriate.
When calling some functions or assigning values to certain attributes,
the arguments/right operand do not match the exact type that the
functions/attributes expect, and thus we fix that by converting types
beforehand.
2021-07-26 15:09:07 -07:00
PIG208 cd678232bd analytics: Extract request.user for typecheck. 2021-07-26 14:48:45 -07:00
PIG208 7d1c475f69 typing: Use assertions for function arguments.
Utilize the assert_is_not_None helper to eliminate errors of
'Argument x to "Foo" has incompatible type "Optional[Bar]"...'
2021-07-26 14:48:45 -07:00
PIG208 8a91d1c2b1 analytics: Fix type annotation for keys.
realm_data, user_data, stream_data and installation_data allow the key to
be int, str, or None.
This fix the type annotation for these dictionaries.
2021-07-26 14:48:45 -07:00
PIG208 df1bf9e352 analytics: Fix type annotation for sql_data_collector. 2021-07-26 14:46:45 -07:00
PIG208 66b1a4e7ca backend: Add None-checks with assertions and if-elses.
This fixes a batch of mypy errors of the following format:
'Item "None" of "Optional[Something]" has no attribute "abc"'
2021-07-24 15:00:21 -07:00
PIG208 495a8476be tests: Use assertion to enforce None-checks in tests.
This fixes a batch of mypy errors of the following format:
'Item "None" of "Optional[Something]" has no attribute "abc"

Since we have already been recklessly using these attritbutes
in the tests, adding assertions beforehand is justified presuming
that they oughtn't to be None.
2021-07-24 09:54:21 -07:00
Anders Kaseorg fb3ddf50d4 python: Fix mypy no_implicit_reexport errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-16 14:02:31 -07:00
Tim Abbott 4d0611c363 support: Add test for get_org_type_display_name.
This apparently lost coverage while I was integrating
faa695e86d.
2021-07-15 11:11:23 -07:00
Eeshan Garg 366b7fe2f5 analytics: Display org_type in /activity/support results. 2021-07-15 10:14:39 -07:00
Eeshan Garg 434c262365 analytics: Display org_type on /activity.
Moving forward we are hoping to collect data on org types from our
users, so it makes sense to display the org type on the "Counts"
tab of our /activity page.
2021-07-15 10:14:38 -07:00
akshatdalton e203112fd4 refactor: Use `assert_length` helper instead of `assertTrue/assertEqual`. 2021-07-13 13:03:38 -07:00
Tim Abbott c804ab27d5 actions: Rename do_activate_user to do_activate_mirror_dummy_user.
This function had a confusing name, which could result in someone
using it unintentionally when they meant do_reactivate_user.

We also add docstrings for both functions.
2021-07-08 17:33:17 -07:00
PIG208 dcbb2a78ca python: Migrate most json_error => JsonableError.
JsonableError has two major benefits over json_error:
* It can be raised from anywhere in the codebase, rather than
  being a return value, which is much more convenient for refactoring,
  as one doesn't potentially need to change error handling style when
  extracting a bit of view code to a function.
* It is guaranteed to contain the `code` property, which is helpful
  for API consistency.

Various stragglers are not updated because JsonableError requires
subclassing in order to specify custom data or HTTP status codes.
2021-06-30 16:22:38 -07:00
Tim Abbott e0c7581168 analytics: Rename get_activity to get_installation_activity. 2021-06-17 17:39:28 -07:00
Tim Abbott 0e358e4cc2 analytics: Rename analytics/views/legacy.py. 2021-06-17 17:39:28 -07:00
Tim Abbott e083134764 analytics: Extract analytics/views/user_activity.py. 2021-06-17 17:39:28 -07:00
Tim Abbott 0d2e5cc390 analytics: Extract analytics/views/realm_activity.py. 2021-06-17 17:39:28 -07:00
Tim Abbott 7099dff91a analytics: Extract common /activity views functions.
This will be helpful for further splitting the giant files here.
2021-06-17 17:39:28 -07:00
Tim Abbott d22acb23bc analytics: Extract analytics/views/support.py. 2021-06-17 17:39:28 -07:00
Tim Abbott 5d8613a02c analytics: Extract analytics/views/stats.py.
This is much higher quality code than some of the older /activity
code, and it's nice to give it its own home.
2021-06-17 17:39:28 -07:00
Tim Abbott ee54ecfccf analytics: Split test_views.py.
This will aligns the test organization with an upcoming reorganization
of the code itself.
2021-06-17 17:39:28 -07:00
Tim Abbott 5891849b0d analytics: Extract test_stats_views.
This will help make the analytics views codebase more readable.
2021-06-17 17:39:28 -07:00
Tim Abbott 9c8ddef5f2 analytics: Create analytics/views/ directory. 2021-06-17 17:39:28 -07:00
Tim Abbott c36b55e418 analytics: Remove stream_stats management command.
We will eventually provide features like this in the /stats UI, and in
any case this doesn't work and hasn't been maintained in years.
2021-06-17 11:30:42 -07:00
Vishnu KS 4e26746b98 tests: Move tests of /activity pages from zerver to analytics module.
This module deals with the testing of /activity, /realm_activity
and /user_activity. All these pages reside in analytics module.
Keeping these tests in zerver/tests is kind is not appropriate
since person who makes changes to /activity pages would not think
it is necessary to run tests in zerver. So better to keep them
in the analytics module.
2021-06-11 07:41:02 -07:00
Vishnu KS d32de76c23 activity: Rename amount variables to arr. 2021-06-11 07:41:02 -07:00
Vishnu KS cfbea80f08 activity: Don't show plan, ARR and discount when billing is disabled. 2021-06-11 07:41:02 -07:00
Vishnu KS 40ab415005 activity: Show effective rate of realms in /activity page. 2021-06-11 07:41:02 -07:00
Vishnu KS 8c055107d9 exceptions: Move InvitationError to zerver/lib/exceptions. 2021-06-09 17:42:38 -07:00
Abhijeet Prasad Bodas 352634a851 tests: Consistently use assert_length helper.
This helper does some nice things like printing out
the data structure incase of failure.
2021-05-19 11:55:56 -07:00
Anders Kaseorg dd3fa4ac52 templates: Mark all void tags as self-closing.
This reverses the policy that was set, but incompletely enforced, by
commit 951514dd7d.  The self-closing tag
syntax is clearer, more consistent, simpler to parse, compatible with
XML, preferred by Prettier, and (most importantly now) required by
FormatJS.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-21 09:49:34 -07:00
Vishnu KS ab771e4b19 support: Show the first human user in realm search result. 2021-04-16 13:22:02 -07:00
Vishnu KS 8139896e3d support: Show realm owners in realm search results. 2021-04-16 13:18:46 -07:00
Vishnu KS 97765798d7 support: Rename realm_admin_emails to get_realm_admin_emails_as_string. 2021-04-16 13:18:46 -07:00
Anders Kaseorg e7ed907cf6 python: Convert deprecated Django ugettext alias to gettext.
django.utils.translation.ugettext is a deprecated alias of
django.utils.translation.gettext as of Django 3.0, and will be removed
in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Tim Abbott 6b1ca15ba7 activity: Use realm owners, not realm administrators.
The analytics system was not properly updated when we replaced realm
owners with realm administrators.
2021-04-08 17:47:23 -07:00
Siddharth Asthana 47e478945b support: Create RealmAuditLog when updating realm_subdomain. 2021-04-03 08:01:46 -07:00
Siddharth Asthana 80c9243c6a support: Create RealmAuditLog when updating sponsorship status. 2021-04-03 08:01:46 -07:00
Siddharth Asthana ddbc6d7662 support: Create RealmAuditLog when realm reactivation email is sent. 2021-04-03 08:01:46 -07:00
Siddharth Asthana 233c4d520c support: Create RealmAuditLog when updating billing_method.
This commit also makes acting_user as a mandantory argument and fixes
the tests accordingly.
2021-04-03 08:01:46 -07:00
Siddharth Asthana 6945ed3587 support: Pass acting_user to attach_discount_to_realm. 2021-04-03 08:01:46 -07:00
Siddharth Asthana c3f37c2a64 support: Pass acting_user to do_change_plan_type.
acting_user is now a mandatory field, so wherever this function is used,
we are passing acting_user as well.
2021-04-03 08:01:46 -07:00
shanukun 88262a484c support: Fix arguments of timesince for expires_in.
`expires_in` (remaining time before the invite expires) should
be calculated from the time at present, not from the time when
confirmation link was sent.
2021-04-02 22:01:57 -07:00
shanukun 4b00e5da72 refactor: Make acting_user a mandatory kwarg for do_deactivate_realm. 2021-04-02 14:44:41 -07:00
Mateusz Mandera f329878376 migrations: Subscription.is_user_active denormalization - step one.
This adds the is_user_active with the appropriate code for setting the
value correctly in the future. In the following commit a migration to
backfill the value for existing Subscriptions will be added.

To ensure correct user_profile.is_active handling also in tests, we
replace all direct .is_active mutation with calls to appropriate
functions.
2021-03-30 09:19:03 -07:00
shanukun c95061e9b9 refactor: Make acting_user a mandatory kwarg for do_deactivate_user. 2021-03-29 15:51:45 -07:00
shanukun 8f3ae715c0 refactor: Make acting_user a mandatory kwarg for do_reactivate_user. 2021-03-29 15:51:45 -07:00
shanukun 459710a897 refactor: Make acting_user a mandatory kwarg for do_set_realm_property. 2021-03-29 15:51:45 -07:00
Tim Abbott 4bdaec7a8d analytics: Stop overriding debug_mode.
This had the same value as the default in default_page_params, and
appears to have been present since this page was introduced in
b26c38bc47.
2021-03-26 09:41:12 -07:00
Vishnu KS 1a126f8bf2 tests: Don't harcode email addresses in analytics.tests.test_views. 2021-03-22 18:10:03 -07:00
Vishnu KS 79c4a25d20 tests: Create assert_user_details_in_html_response helper function. 2021-03-22 18:10:03 -07:00
Vishnu KS a404711740 support: Show the actual user email in /support. 2021-03-22 18:10:03 -07:00
Vishnu KS 38e7903425 support: Show role of the user in search result. 2021-03-22 18:10:03 -07:00
aryanshridhar 04fb0552a6 analytics: escape HTML correctly when generating links.
Wrapped the html text within html.escape function to convert
special characters into HTML-safe string while generating link.
2021-03-18 15:08:31 -07:00
Tim Abbott 0da1bd43e9 analytics: Remove buggy HttpResponseNotFound text.
Had this been in normal route, this would have been an XSS bug, as we
were passing what the developer clearly believed to be plain text into
an HTML 404 page.

The affected routes have @require_server_admin, a permission that we
do not expect any self-hosted users to have ever enabled (as it is
undocumented and doing so is only possible manually via a `manage.py
shell`, and we believe to only be useful for running a SaaS service
like zulip.com).  So the security impact is limited to a handful of
staff of zulip.com and this isn't a candidate for a CVE.

Thanks to GitHub's CodeQL for finding this.
2021-03-18 12:16:15 -07:00
Mateusz Mandera d91d3a05b9 tests: Use do_create_realm where possible.
Using do_create_realm should be preferred over manual creation where
possible, as it creates more realistic data.
2021-03-14 08:50:02 -07:00
shanukun fafe1a31d7 refactor: Make acting_user a mandatory kwarg for do_activate_user. 2021-02-25 17:58:00 -08:00
shanukun 4b67946605 refactor: Make acting_user a mandatory kwarg for do_create_user. 2021-02-25 17:58:00 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 5028c081cb python: Merge concatenated string literals that Black would uglify.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 9773c0f1a8 python: Fix string literal concatenation mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:02:51 -05:00
Siddharth Asthana 6c888977a6 change_subdomain: Create a deactivated realm on updating subdomain.
When changing the subdomain of a realm, create a deactivated realm with
the old subdomain of the realm, and set its deactivated_redirect to the
new subdomain.
Doing this will help us to do the following:
- When a user visits the old subdomain of a realm, we can tell the user
that the realm has been moved.
- During the registration process, we can assure that the old subdomain
of the realm is not used to create a new realm.

If the subdomain is changed multiple times, the deactivated_redirect
fields of all the deactivated realms are updated to point to the new
uri.
2021-01-07 14:15:22 -08:00
Vishnu KS 9fe39646fa analytics: Specify exact end_time in realm summary query.
Fetchings rows with end_time within the last 25 hours would result
in the realmcount queries returning two rows for each realm
if the analytics page was opened within an hour since the
count stats were updated.
2020-12-22 16:44:31 -08:00
Vishnu KS 9d5a1271d4 analytics: Make last_successful_fill handle FillState.STARTED case properly.
Subtracting an hour from end_time is correct only for CountStats with
hourly frequency. For daily frequency we should subtract a day instead.
2020-12-22 16:44:31 -08:00
Vishnu KS 235a347639 analytics: Move last_successful_fill to CountStat.
This is a prep commit. Currenty we only pass CountStat.property
to last_successful_fill function. But it needs access to
CountStat.time_increment as well. We can pass the entire CountStat
object to the function as a workaround. But making last_successful_fill
a property of CountStat seems to be much more cleaner.
2020-12-22 16:44:31 -08:00
Vishnu KS 189e9a2759 analytics: Create time_increment property in CountStat. 2020-12-22 16:44:31 -08:00
Vishnu KS 7b2f16bc5c support: Mention % in attach discount success message. 2020-12-17 17:09:20 -08:00
Vishnu KS dd5bcb97e8 support: Allow user look up through full name search. 2020-12-17 17:07:28 -08:00
Siddharth Asthana 6f962c1815 support: Add ability to change subdomain of realms. 2020-12-06 00:48:10 -08:00
Siddharth Asthana 37e158b9ba support: Rename message -> success_message. 2020-12-06 00:48:10 -08:00
m-e-l-u-h-a-n cbfd6464a5 logging: replace mock.patch() for logging with assertLogs()
This commit removes mock.patch with assertLogs().

* Adds return value to do_rest_call() in outgoing_webhook.py, to
  support asserting log output in test_outgoing_webhook_system.py.

* Logs are not asserted in test_realm.py because it would require to users
  to be queried using users=User.objects.filter(realm=realm) and the order
  of resulting queryset varies for each run.

* In test_decorators.py, replacement of mock.patch is not done because
  I'm not sure if it's worth the effort to replace it as it's a return
  value of a function.

Tweaked by tabbott to set proper mypy types.
2020-10-29 15:37:45 -07:00
Vishnu KS f1ceab69c9 activity: Speed up the main query in /activity. 2020-10-23 12:45:20 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Steve Howell 31eb97ddde performance: Fix do_mark_stream_messages_as_read.
This function no longer asks for data that it
doesn't need.
2020-10-16 12:58:11 -07:00
Anders Kaseorg f0ae2a88f2 activity: Show server time rather than browser time.
This gets rid of an inline <script> that was blocking #16451.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-08 19:17:35 -07:00
Abhijeet Prasad Bodas bd61b570aa stats: Fix strings passed to _() not translated.
Part of #16094.
Strings constructed by _() were not being
translated in the /stats page.
This was because session variable was not set.
Ideally this should have been a part of b82bda9.
2020-10-02 14:56:20 -07:00
Vishnu KS 45b7673f4a support: Show realm string_id instead of name in alerts.
Since realm.name can be empty.
2020-09-28 15:37:49 -07:00
Vishnu KS 510efbc1a8 support: Add option to change billing method. 2020-09-28 15:37:49 -07:00
Vishnu KS 3e438538b4 support: Add support for downgrading realm. 2020-09-28 15:37:49 -07:00
Abhijeet Prasad Bodas b82bda9fb8 stats: Fix i18n not working in stats page.
Part of #16094.

Strings tagged with i18n were not being translated on the stats page.
This was because the translation data wasn't being sent to the front
end for this page.  That logic will be required in any page with a
bundle containing i18n JavaScript.
2020-09-25 16:22:34 -07:00
Anders Kaseorg 0228acf0f5 rest: Add rest_path shortcut for path with rest_dispatch.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:51:00 -07:00
Anders Kaseorg 5297e4a30a urls: Use unqualified imports.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:51:00 -07:00
Anders Kaseorg e70f2ae58d rest: Specify rest_dispatch handlers by function, not by string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:46:28 -07:00
Anders Kaseorg faf600e9f5 urls: Remove unused URL names and shorten others.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:46:28 -07:00
Alex Vandiver 536bd3188e middleware: Move locale-setting before domain checking.
Calling `render()` in a middleware before LocaleMiddleware has run
will pick up the most-recently-set locale.  This may be from the
_previous_ request, since the current language is thread-local.  This
results in the "Organization does not exist" page occasionally being
in not-English, depending on the preferences of the request which that
thread just finished serving.

Move HostDomainMiddleware below LocaleMiddleware; none of the earlier
middlewares call `render()`, so are safe.  This will also allow the
"Organization does not exist" page to be localized based on the user's
browser preferences.

Unfortunately, it also means that the default LocaleMiddleware catches
the 404 from the HostDomainMiddlware and helpfully tries to check if
the failure is because the URL lacks a language component (e.g.
`/en/`) by turning it into a 304 to that new URL.  We must subclass
the default LocaleMiddleware to remove this unwanted functionality.

Doing so exposes a two places in tests that relied (directly or
indirectly) upon the redirection: '/confirmation_key'
was redirected to '/en/confirmation_key', since the non-i18n version
did not exist; and requests to `/stats/realm/not_existing_realm/`
incorrectly were expecting a 302, not a 404.

This regression likely came in during f00ff1ef62, since prior to
that, the HostDomainMiddleware ran _after_ the rest of the request had
completed.
2020-09-14 22:16:09 -07:00
Anders Kaseorg 1c80188648 urls: Elide str converter, which is the default.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:21:40 -07:00
Anders Kaseorg a50fae89e2 python: Elide type=str from argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg b4597a8ca8 python: Elide default for store_{true,false} argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg b84260014a populate_analytics_db: Replace intermediate list with generator.
Followup to commit ab120a03bc (#16265).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 19:34:48 -07:00
Anders Kaseorg ab120a03bc python: Replace unnecessary intermediate lists with generators.
Mostly suggested by the flake8-comprehension plugin.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg 1ded51aa9d python: Replace list literal concatenation with * unpacking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Tim Abbott c05315ace2 update_analytics_counts: Fix warning output. 2020-08-28 14:24:48 -07:00
Anders Kaseorg 51f993e084 python: Remove unittest.mock.Mock uses from production code.
It’s somewhat expensive to import and confuses mypy.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-28 11:34:09 -07:00
Anders Kaseorg 61d0417e75 python: Replace ujson with orjson.
Fixes #6507.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:55:12 -07:00
Vishnu KS 4dc83a139c counts: Create 7day_actives::day counstat. 2020-08-10 17:22:19 -07:00
Anders Kaseorg 02fcb75ded analytics: Convert datetime to UNIX timestamp before JSON serialization.
datetime objects are not ordinarily JSON serializable.  While both
ujson and orjson have special cases to serialize datetime objects,
they do it in different ways.  So we want to do this explicitly.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:05:01 -07:00
arpit551 a68d38cc52 migrations: Upgrade migrations to remove duplicates in all Count tables.
This commit upgrades 0015_clear_duplicate_counts migration to remove
duplicate count in StreamCount, UserCount, InstallationCount as well.

Fixes https://github.com/zulip/docker-zulip/issues/266
2020-07-30 15:18:00 -07:00
Vishnu KS 18ad35013f support: Return json error if the POST request is invalid. 2020-07-28 11:03:06 -07:00
Vishnu KS 5b0b1efb15 support: Add functionality to approve sponsorship requests.
This should make it much easier to process these requests.
2020-07-24 17:55:38 -07:00
Vishnu KS 1a1396d07e support: Show customer plan details in support page. 2020-07-24 17:37:41 -07:00
Tim Abbott bcab06509a analytics: Remove unused analytics management commands.
Unlike stream_stats, I'm not aware of any of these having been used in
the last few years, and it's basically just really bad subsets of the
data in /activity, which also doesn't require shell access to use.

These haven't had real work or usage, AFAIK, since 2013.
2020-07-24 13:10:43 -07:00
Vishnu KS 49d06b9d69 activity: Always include realms on standard plan. 2020-07-24 13:09:43 -07:00
Steve Howell c44500175d database: Remove short_name from UserProfile.
A few major themes here:

    - We remove short_name from UserProfile
      and add the appropriate migration.

    - We remove short_name from various
      cache-related lists of fields.

    - We allow import tools to continue to
      write short_name to their export files,
      and then we simply ignore the field
      at import time.

    - We change functions like do_create_user,
      create_user_profile, etc.

    - We keep short_name in the /json/bots
      API.  (It actually gets turned into
      an email.)

    - We don't modify our LDAP code much
      here.
2020-07-17 11:15:15 -07:00
arpit551 87aaa84b42 audit_log: Log acting_user in do_change_user_role. 2020-07-06 17:32:11 -07:00
arpit551 19a8841a9e audit_log: Log acting_user in do_scrub_realm. 2020-07-06 17:24:18 -07:00
Vishnu KS 1e68525f83 support: Ensure that only one form is posted at a time.
The forms to change plan_type, add discount, scrub_realm etc
all post to the same endpoint.

Our frontend code is written so that only one form posts at a time.
But there should be no harm in enforcing the same in backend as well.
2020-07-01 16:45:38 -07:00
Vishnu KS 4c6350fa4b billing: Add option to request a sponsorship in /upgrade. 2020-07-01 16:45:38 -07:00
sahil839 1f8f227444 models: Update values of PreregistrationUser.invite_as dict.
This commit changes the PreregistrationUser.invite_as dict to have
same set of values as we have for UserProfile.role.

This also adds a data migration to update the already exisiting
PreregistrationUser and MultiuseInvite objects.
2020-06-24 11:09:07 -07:00
wowol 035d047dff urls: Migrate analytics urls to use modern django patterns. 2020-06-23 15:02:42 -07:00
Anders Kaseorg 3916ea23a9 python: Combine some split import groups.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-18 15:54:11 -07:00
Anders Kaseorg f364d06fb5 python: Convert percent formatting to .format for translated strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-15 16:24:46 -07:00
Anders Kaseorg 5dc9b55c43 python: Manually convert more percent-formatting to f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 74c17bf94a python: Convert more percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Now including %d, %i, %u, and multi-line strings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 4b6d2cf25f logging: Pass more format arguments to logging.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 1a3441dbf5 confirmation: Pass realm rather than host to confirmation_url.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
arpit551 aa70baba71 analytics: Add backend for messages read over time graph.
This commit includes changes in the api /json/analytics/chart_data to
send data for the newly added graph, as well as tests.
2020-06-14 21:19:24 -07:00
arpit551 a4b857b635 analytics: Populate data for messages_read::hour in development.
Generate data for messages_read::hour so that we can test messages_
read_over_time graph.
2020-06-14 21:19:14 -07:00
arpit551 c4b5d09283 analytics: Add LoggingCount for messages read stats.
Whenever we use API queries to mark messages as read we now increment
two new LoggingCount stats, messages_read::hour and
messages_read_interactions::hour.

We add an early return in do_increment_logging_stat function if there
are no changes (increment is 0), as an optimization to avoid
unnecessary database queries.

We also log messages_read_interactions::hour Logging stat
as the number of API queries to mark messages as read.

We don't include tests for the case where do_update_pointer is called
because do_update_pointer will most likely be removed from the
codebase in the near future.
2020-06-14 21:15:27 -07:00
arpit551 27daf38587 analytics: Derive AnalyticsTestCase class from ZulipTestCase.
The ZulipTestCase class contains various useful utility functions that
we could be using in test_counts.
2020-06-14 21:08:24 -07:00
Anders Kaseorg 0d6c771baf python: Guard against default value mutation with read-only types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 69c0959f34 python: Fix misuse of Optional types for optional parameters.
There seems to have been a confusion between two different uses of the
word “optional”:

• An optional parameter may be omitted and replaced with a default
  value.
• An Optional type has None as a possible value.

Sometimes an optional parameter has a default value of None, or None
is otherwise a meaningful value to provide, in which case it makes
sense for the optional parameter to have an Optional type.  But in
other cases, optional parameters should not have Optional type.  Fix
them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg 67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg 6480deaf27 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, with more
restrictions patched out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 14:48:09 -07:00
Anders Kaseorg 5839fdf963 analytics: Improve escaping correctness with psycopg2.sql.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 21:12:43 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
arpit551 fb2aae1c02 analytics tests: Save recipient in stream object.
At the time of creating streams in test_counts.py we earlier did not saved
recipient in the stream object.

stream.recipient is used in many functions so they would throw error.

The right long-term fix here is probably to just use the standard
stream creation functions rather than having a hacky duplicate
here.
2020-06-08 11:33:24 -07:00
Anders Kaseorg 1f565a9f41 timezone: Use standard library datetime.timezone.utc consistently.
datetime.timezone is available in Python ≥ 3.2.  This also lets us
remove a pytz dependency from the PostgreSQL scripts.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 09:34:17 -07:00
Sahil Batra 77d4be56a4
users: Modify do_create_user and create_user to accept role.
We change do_create_user and create_user to accept
role as a parameter instead of 'is_realm_admin' and 'is_guest'.
These changes are done to minimize data conversions between
role and boolean fields.
2020-06-02 16:11:36 -07:00
Anders Kaseorg 840cf4b885 requirements: Drop direct dependency on mock.
mock is just a backport of the standard library’s unittest.mock now.

The SAMLAuthBackendTest change is needed because
MagicMock.call_args.args wasn’t introduced until Python
3.8 (https://bugs.python.org/issue21269).

The PROVISION_VERSION bump is skipped because mock is still an
indirect dev requirement via moto.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-26 11:40:42 -07:00
sahil839 1aebf3cab9 actions: Merge do_change_is_admin and do_change_is_guest.
This commit merges do_change_is_admin and do_change_is_guest to a
single function do_change_user_role which will be used for changing
role of users.

do_change_is_api_super_user is added as a separate function for
changing is_api_super_user field of UserProfile.
2020-05-25 16:17:10 -07:00
sahil839 9b78a73e36 populate_db: Add new admin user as 'Desdemona'.
This commit adds a second admin user named 'Desdemona' to dev and
test database.
2020-05-19 11:42:27 -07:00
sahil839 36dca5ba5c analytics: Add order_by to query used for fetching admin emails.
This commit adds order_by to the query that fetches admin emails
in views.py. This is added to show emails alphabetically in case
of multiple admins.
2020-05-19 11:42:27 -07:00
Anders Kaseorg 8cdf2801f7 python: Convert more variable type annotations to Python 3.6 style.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-08 16:42:43 -07:00
Pragati Agrawal bd9b74436c org settings: Enable message_retention_days in org settings UI.
Since production testing of `message_retention_days` is finished, we can
enable this feature in the organization settings page. We already had this
setting in frontend but it was bit rotten and not rendered in templates.

Here we replaced our past text-input based setting with a
dropdown-with-text-input setting approach which is more consistent with our
existing UI.

Along with frontend changes, we also incorporated a backend change to
handle making retention period forever. This change introduces a new
convertor `to_positive_or_allowed_int` which only allows positive integers
and an allowed value for settings like `message_retention_days` which can
be a positive integer or has the value `Realm.RETAIN_MESSAGE_FOREVER` when
we change the setting to retain message forever.

This change made `to_not_negative_int_or_none` redundant so removed it as
well.

Fixes: #14854
2020-05-08 14:09:31 -07:00
Anders Kaseorg bdc365d0fe logging: Pass format arguments to logging.
https://docs.python.org/3/howto/logging.html#optimization

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:18:02 -07:00
Tim Abbott 69ae4931c3 migrations: Use django.db.backends.postgresql.schema.
This replaces django.db.backends.postgresql_psycopg2, which has been
an alias to django.db.backends.postgresql since Django 1.9.
2020-04-26 22:20:24 -07:00
Anders Kaseorg fead14951c python: Convert assignment type annotations to Python 3.6 style.
This commit was split by tabbott; this piece covers the vast majority
of files in Zulip, but excludes scripts/, tools/, and puppet/ to help
ensure we at least show the right error messages for Xenial systems.

We can likely further refine the remaining pieces with some testing.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-    invoiced_through: Optional[LicenseLedger] = models.ForeignKey(
+    invoiced_through: Optional["LicenseLedger"] = models.ForeignKey(

-_apns_client: Optional[APNsClient] = None
+_apns_client: Optional["APNsClient"] = None

-    notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
-    signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)

-    author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)
+    author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)

-    bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)
+    bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)

-    default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
-    default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)

-descriptors_by_handler_id: Dict[int, ClientDescriptor] = {}
+descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {}

-worker_classes: Dict[str, Type[QueueProcessingWorker]] = {}
-queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {}
+worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {}
+queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {}

-AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None
+AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 11:02:32 -07:00
Anders Kaseorg f8c95cda51 mypy: Add specific codes to type: ignore annotations.
https://mypy.readthedocs.io/en/stable/error_codes.html

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 10:46:33 -07:00
Anders Kaseorg 1cf63eb5bf python: Whitespace fixes from autopep8.
Generated by autopep8, with the setup.cfg configuration from #14532.
I’m not sure why pycodestyle didn’t already flag these.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-21 17:58:09 -07:00
Anders Kaseorg c734bbd95d python: Modernize legacy Python 2 syntax with pyupgrade.
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-09 16:43:22 -07:00
Vishnu KS dae5f54a63 stats: Show info message in /stats page of newly created realms.
This provides a better user experience for folks joining a new Zulip
realm and visiting this page.
2020-04-03 11:59:18 -07:00
Vishnu KS 905f3d2d23 stats: Don't show analytics data unavailable error for new realms.
The value of start would be set to realm creation date or installation
date unless a value is explicitly provided by the user. We don't
want to show analytics data missing error in both these cases if
start is less than 24.5 hours from timezone_now() since some of the
stats are populated by a daily cron job that may take a couple of
minutes to run.

The same case applies if the value of the start is passed in the
request.
2020-04-03 11:51:20 -07:00
Steve Howell 1306239c16 tests: Use email/delivery_email more explicitly.
We try to use the correct variation of `email`
or `delivery_email`, even though in some
databases they are the same.

(To find the differences, I temporarily hacked
populate_db to use different values for email
and delivery_email, and reduced email visibility
in the zulip realm to admins only.)

In places where we want the "normal" realm
behavior of showing emails (and having `email`
be the same as `delivery_email`), we use
the new `reset_emails_in_zulip_realm` helper.

A couple random things:

    - I fixed any error messages that were leaking
      the wrong email

    - a test that claimed to rely on the order
      of emails no longer does (we sort user_ids
      instead)

    - we now use user_ids in some place where we used
      to use emails

    - for IRC mirrors I just punted and used
      `reset_emails_in_zulip_realm` in most places

    - for MIT-related tests, I didn't fix email
      vs. delivery_email unless it was obvious

I also explicitly reset the realm to a "normal"
realm for a couple tests that I frankly just didn't
have the energy to debug.  (Also, we do want some
coverage on the normal case, even though it is
"easier" for tests to pass if you mix up `email`
and `delivery_email`.)

In particular, I just reset data for the analytics
and corporate tests.
2020-03-19 16:04:03 -07:00
Mateusz Mandera b4ce167a88 models: Add recipient foreign key to Huddle.
This follows the already tested approach from
8acfa17fe6.
2020-03-17 05:41:11 -07:00
Steve Howell 1b16693526 tests: Limit email-based logins.
We now have this API...

If you really just need to log in
and not do anything with the actual
user:

    self.login('hamlet')

If you're gonna use the user in the
rest of the test:

    hamlet = self.example_user('hamlet')
    self.login_user(hamlet)

If you are specifically testing
email/password logins (used only in 4 places):

    self.login_by_email(email, password)

And for failures uses this (used twice):

    self.assert_login_failure(email)
2020-03-11 17:10:22 -07:00
Mateusz Mandera 64b85415f5 migrations: Fix unused import error. 2020-03-06 12:17:19 -08:00
arpit551 f299f31340 analytics: Fix missing unique constraint when subgroup is null.
Replaced unique_together with UniqueConstraint in models that
covered nullable fields as in unique_together database indexes
don't work where subgroup=None. So added conditional unique
index handling invalid duplicate Count data.

Added 0015_clear_duplicate_counts migration to handle existing
data that violates the constraints.

Also corrected a test case in test_counts.py which didn't clear its
state properly and thus was accidentally taking advantage of this
database schema bug.
2020-03-06 11:10:04 -08:00
arpit551 2899e44218 analytics: Added comments.
A few comments are added to explain more clearly the changes made in
b23a5431cd namely about not using realm
arguments in LoggingCount Stats and the need to pass realm argument in
pull function.

The comments were tweaked by tabbott for readability.
2020-01-28 14:57:32 -08:00
Tim Abbott 9ac3e1099c analytics: Remove last_modified field from FillState.
This field wasn't used for anything, and I think it has very limited
use for debugging, since fundamentally, it'll almost always have a
value within the hour of the actual timestamp in FillState, and any
more fine-grained logging we might want would be available in the
analytics job's own logs.

The proximal reason to remove it is that apparently Django's
model_to_dict doesn't support auto_now fields, and that caused some
trouble when working on adding more complete import/export support for
analytics data.
2020-01-26 20:38:26 -08:00
arpit551 b23a5431cd analytics: Add realm argument to analytics.
This changeset is prepartory work for doing something reasonable with
analytics data during the zulip -> zulip data import process (and
potentially e.g. slack -> Zulip as well).

To support that, we need to make it possible to do our analytics
calculations for a single realm.

We do this while maintaining backwards compatibility and avoiding
massive duplicated code by adding an optional `realm` argument to the
entrypoints to the analytics system, especially process_count_stat.

More work involving restructuring FillState will be required for this
to be actually usable for its intented purpose, but this commit is a
nice checkpoint along the way.

Tweaked by tabbott to adjust comments and disable InstallationCount
updates when a realm argument is specified.
2020-01-23 17:36:13 -08:00
Tim Abbott 8e7ce7cc79 python: Sort migrations/management command imports with isort.
This is a preparatory commit for using isort for sorting all of our
imports, merging changes to files where we can easily review the
changes as something we're happy with.

These are also files with relatively little active development, which
means we don't expect much merge conflict risk from these changes.
2020-01-14 13:07:47 -08:00
Mateusz Mandera 8acfa17fe6 models: Add recipient foreign key in UserProfile and Stream.
This is adds foreign keys to the corresponding Recipient object in the
UserProfile on Stream tables, a denormalization intended to improve
performance as this is a common query.

In the migration for setting the field correctly for existing users,
we do a direct SQL query (because Django 1.11 doesn't provide any good
method for doing it properly in bulk using the ORM.).

A consequence of this change to the model is that a bit of code needs
to be added to the functions responsible for creating new users (to
set the field after the Recipient object gets created).  Fortunately,
there's only a few code paths for doing that.

Also an adjustment is needed in the import system - this introduces a
circular relation between Recipient and UserProfile. The field cannot be
set until the Recipient objects have been created, but UserProfiles need
to be created before their corresponding Recipients. We deal with this
by first importing UserProfiles same way as before, but we leave the
personal_recipient field uninitialized. After creating the Recipient
objects, we call a function to set the field for all the imported users
in bulk.

A similar change is made for managing Stream objects.
2019-12-09 15:14:41 -08:00
Vishnu Ks a26b379a14 support: Send confirmation email on realm activation. 2019-12-02 09:51:45 -08:00
Anders Kaseorg 0d20145b93 mypy: Upgrade from 0.730 to 0.740.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Vishnu KS ec955f8f78 support: Show confirmation links in search.
Fixes #13060 #12784
2019-10-21 16:56:50 -07:00
Vishnu KS 139ebf387b support: Pass various realm functions as template context.
We currently have code to calculate the value of realm_icon_url,
admin_emails and default_discount in two diffrent places. With
the addition of showing confirmation links it would become three.
The easiest way to deduplicate the code and make the view cleaner
is by doing the calculations in template. Alternatively one can
write a function that takes users, realms and confirmations as
arguments and sets the value of realm_icon_url, admin_emails and
default_discount appropriately in realm object according to the
type of the confirmation. But that seems more messy than passing
the functions directly to template approach.
2019-10-21 16:52:46 -07:00
Mateusz Mandera bbf2474bd0 tests: setUp overrides should call super().setUp().
MigrationsTestCase is intentionally omitted from this, since migrations
tests are different in their nature and so whatever setUp()
ZulipTestCase may do in the future, MigrationsTestCase may not
necessarily want to replicate.
2019-10-19 17:27:01 -07:00
Rishi Gupta e10361a832 models: Replace is_guest and is_realm_admin with UserProfile.role.
This new data model will be more extensible for future work on
features like a primary administrator.
2019-10-06 16:24:37 -07:00
Rishi Gupta 4256ee61cf billing: Change RealmAuditLog.event_type from str to int.
This is a more robust long-term model for storing these data.
2019-10-06 15:55:56 -07:00
Mateusz Mandera dbe508bb91 models: Migration of Message.pub_date to date_sent, part 2.
Fixes #1727.

With the server down, apply migrations 0245 and 0246. 0246 will remove
the pub_date column, so it's essential that the previous migrations
ran correctly to copy data before running this.
2019-10-05 19:01:34 -07:00
Tim Abbott a352f2e10d test_counts: Remove custom user creation code.
Like the last commit, this avoids us needing to update this random
test class in the analytics subsystem when we adjust the UserProfile
model.
2019-09-19 14:31:58 -07:00
Tim Abbott aae610f65c analytics: Fix creation of shylock user.
Previously, the shylock user was created using a bad duplicate of our
standard create_user code path; we fix this by just calling into that
code path.
2019-09-19 14:16:38 -07:00
Anders Kaseorg 7494f1600c templates: Move page_params from an inline script to the <body> dataset.
This sidesteps tricky escaping issues, and will make it easier to
build a strict Content-Security-Policy.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-17 16:06:33 -07:00
Anders Kaseorg 490ce993f4 analytics: Clean up type ignores.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 16:39:16 -07:00
Anders Kaseorg becef760bf cleanup: Delete leading newlines.
Previous cleanups (mostly the removals of Python __future__ imports)
were done in a way that introduced leading newlines.  Delete leading
newlines from all files, except static/assets/zulip-emoji/NOTICE,
which is a verbatim copy of the Apache 2.0 license.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-06 23:29:11 -07:00
Wyatt Hoodes 97c10b35c2 analytics/views: Remove redundant check of `if end is None`. 2019-07-31 12:15:57 -07:00
Wyatt Hoodes fa227d79af typing: Fix options typing in management/commands.
We simply state that certain options are `Optional`.

The following files are affected:

add_users_to_mailing_list
send_to_email_mirror
fill_memcached_caches
client_activity
2019-07-31 12:13:56 -07:00
Mateusz Mandera b321dcd50b test_views: Prepare for moving system bots to zulipinternal. 2019-07-24 16:26:10 -07:00
Rishi Gupta 9fc791926b support: Use delivery_email instead of email.
Structurally, these pages want email addresses one can actually send
emails to.
2019-07-21 14:47:15 -07:00
Tim Abbott 4eb9d67b70 models: Extract get_human_admin_users function.
This function is an alternative to get_admin_users that we use in all
places where we explicitly want only human administrative users (not
administrative bots).  The following commits will rename
get_admin_users for better clarity.
2019-06-20 14:32:30 -07:00
Mohit Gupta db3d81613b decorator: Refactor @require_non_guest_human_user decorator.
Rename @require_non_guest_human_user to @require_member_or_admin.
This is a refactor commit prior to introduction of Administrator
Bots.
2019-06-18 17:11:58 -07:00
Vishnu Ks 14e582fb59 support: Add functionality to copy admin emails.
Also renamed a bunch of functions in test_views for better
readability.
2019-06-14 10:19:50 -07:00
Roman Godov a50824e031 models: Rename Subscription.in_home_view field to is_muted.
This renames Subscription.in_home_view field to is_muted, for greater
clarity as to what it does just from seeing the setting name, without
having to look it up.

Also disabled an obsolete test_migrations test.

Fixes #10042.
2019-05-12 22:08:10 -07:00
Tim Abbott 1d44fd724b audit log: Log which server admin deactivated a realm too. 2019-05-08 15:09:48 -07:00
Rishi Gupta 0218da64f1 support: Update success message for plan type change. 2019-05-08 15:09:48 -07:00
Rishi Gupta 98da11c558 support: Rename deactive to deactivated. 2019-05-08 15:09:48 -07:00
Anders Kaseorg 9efda71a4b get_realm: raise DoesNotExist instead of returning None.
This makes the implementation of `get_realm` consistent with its
declared return type of `Realm` rather than `Optional[Realm]`.

Fixes #12263.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-06 21:58:16 -07:00
Vishnu Ks 6c58603eaf support: Add support for scrubbing realm. 2019-05-06 20:12:54 -07:00
Vishnu Ks f6203f068b support: Add support for activating and deactivating realm. 2019-05-06 20:12:48 -07:00
Vishnu Ks 123bcea518 management: Don't use sys.exit(1).
Using sys.exit in a management command makes it impossible
to unit test the code in question.  The correct approach to do the same
thing in Django management commands is to raise CommandError.

Followup of b570c0dafa
2019-05-03 14:20:39 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00
Vishnu Ks 8eeb8280b4 activity: Create interface for doing support operations.
This should grow into a tool that makes it much easier to do common
organization management tasks without using a manage.py shell.
2019-03-11 12:01:11 -07:00
Vishnu Ks 42de9a0c71 analytics: Extract out the function for getting plan name. 2019-03-11 12:01:11 -07:00
Harshit Bansal 0e401c4f18 minor: Fix an artifact of delivery email migration in `populate_analytics_db`. 2019-03-05 13:52:59 -08:00
Rishi Gupta 9962377018 analytics: Fix midnight-related bug in test.
Previously, this would flake if the day changed between
user2 = do_create_user('email2', 'password', ...) and
do_deactivate_user(user2).
2019-02-28 14:48:30 -08:00
Tim Abbott 216d2ec1bf production: Add optional support for submitting usage statistics.
See documentation for details.
2019-02-26 17:35:10 -08:00
Tim Abbott c41bfcb9e0 Revert "activity: Change definition of active site."
This reverts commit 9f9b7cb991.

This commit made the page not perform well enough to load.
2019-02-13 14:52:13 -08:00
bartek 9f9b7cb991 activity: Change definition of active site.
Signed-off-by: bartek <bartek.jachowicz@gmail.com>

Edits by Rishi Gupta <rishig@zulipchat.com>

Fixes: #10432
2019-02-12 16:36:44 -08:00
Wyatt Hoodes bb6a75c3dc populate_analytics_db.py: Subscribe user for exporting.
AssertionErrors were raised when attempting to run manual comparison
tests to ensure correctness when exporting the analytics realm using
export_from_config.  This was caused by this populate_analytics_db
stream being created without any subscribers, which violates an
invariant.

We fix this by simply subscribing the 'shylock' user to that stream.
2019-02-04 10:59:24 -08:00
Anders Kaseorg c2cb804ebe analytics/views.py: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:43:58 -08:00
Anders Kaseorg f5197518a9 analytics/zilencer/zproject: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:31:45 -08:00
Tim Abbott 13b9e90a40 analytics: Fix importing zilencer.
In ebdd55814c, we added zilencer imports
without using the proper mocking procedure for when zilencer is not enabled.

This whole setup is a mess and probably we should enable zilencer
unconditionally in a future version.
2019-02-02 17:18:57 -08:00
Anders Kaseorg 4bd28f7ae6 migrations: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:01:04 -08:00
Tim Abbott 96f096f38e analytics: Add section for remote Zulip servers. 2019-02-02 16:55:12 -08:00
Tim Abbott ebdd55814c analytics: Add support for graphing remote analytics data.
Combined with recent work on submitting data, one can now view the
high-level usage numbers for chat.zulip.org on zulipchat.com's
dashboards.
2019-02-02 16:55:12 -08:00
Rishi Gupta 85f7ac8172 analytics: Remove Anomaly model. 2019-02-01 18:48:18 -08:00
ss62171 5649b954ef stream_stats: Add a column representing type of stream.
This adds a column which represents whether a stream is public or
private.

Fixes #11374.
2019-01-31 15:04:45 -08:00
ss62171 5fd9748e13 stream_stats: List number of private and public streams for each realm.
Previously, we only displayed public streams and didn't list the
number of them.  Now, we just list all streams.
2019-01-31 15:04:09 -08:00
Rishi Gupta e7220fd71f billing: Do subscription management in-house instead of with Stripe Billing.
This is a major rewrite of the billing system. It moves subscription
information off of stripe Subscriptions and into a local CustomerPlan
table.

To keep this manageable, it leaves several things unimplemented
(downgrading, etc), and a variety of other TODOs in the code. There are also
some known regressions, e.g. error-handling on /upgrade is broken.
2018-12-22 13:39:30 -08:00
Tim Abbott e603237010 email: Convert accounts code to use delivery_email.
A key part of this is the new helper, get_user_by_delivery_email.  Its
verbose name is important for clarity; it should help avoid blind
copy-pasting of get_user (which we'll also want to rename).
Unfortunately, it requires detailed understanding of the context to
figure out which one to use; each is used in about half of call sites.

Another important note is that this PR doesn't migrate get_user calls
in the tests except where not doing so would cause the tests to fail.
This probably deserves a follow-up refactor to avoid bugs here.
2018-12-06 16:21:38 -08:00
Tim Abbott 7ddcbd0d3a analytics: Set delivery_email in hacky test user creation code. 2018-12-06 15:33:28 -08:00
Tim Abbott c679920c01 python: Fix unnecessary uses of str_utils library. 2018-11-27 11:44:09 -08:00
Rishi Gupta 91b02373dc activity: Rename standard free to open source.
"standard free" looks too similar to "standard". Also it makes it hard to
Ctr+f for "standard".
2018-11-16 21:03:37 -08:00
Vishnu Ks 2e04cdbe5e billing: Show estimated subscription revenue on /activity.
[Substantial edits by Rishi Gupta]
2018-11-16 13:30:16 -08:00
Vishnu Ks 9d398ee335 activity: Move the total row from last to first. 2018-11-16 09:22:32 -08:00
Steve Howell c0fd8660d2 subject -> topic: Fix analytics test (minor). 2018-11-14 23:24:06 -08:00
Yashashvi Dave 02a5849d4c statistics: Guest user can't access realm statistics.
Don't allow guest user to access realm statistics from
UI or at API level.

Fixes part of #10749.
2018-11-02 11:43:09 -07:00
Rishi Gupta 458169928c billing: Rename Zulip Premium to Zulip Standard. 2018-10-24 10:42:16 -07:00
Tim Abbott 0deeffff6d populate_analytics_db: Make shylock an organization admin.
This means the analytics realm satisfies the usual Zulip invariant
that every realm has at least one organization admin.
2018-08-31 15:26:15 -07:00
Vishnu Ks 6fcb095e70 activity: Show plan_type in activity page. 2018-08-21 18:15:14 -07:00
Roman Godov 34ae3dfd44 models: Delete unused Subscription.notifications field.
This deletes the unused Subscription.notifications field and removes
it from some testing and analytics code (which should not have been
using it in the first place).

Fixes #10042.
2018-07-26 15:54:57 -07:00
Anders Kaseorg 8d52f0e0c0 analytics/management/commands/check_analytics_state.py: Avoid shelling out for mv.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Vishnu Ks 201b99a6f8 models: Add USER_REACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks d0b89cbb44 models: Add USER_DEACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks ce3fffdbb2 models: Add USER_ACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 2c8effe9fe models: Add USER_CREATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 4d1a68430a analytics: Remove unused RealmAuditLog import. 2018-07-10 15:42:26 +05:30
Tim Abbott b9fa7d7b6d populate_analytics_db: Flush memcached after deleting analytics realm.
This fixes a subtle bug where if you reran populate_analytics_db
directly, we'd end up in a weird state where memcached fetched the
"old" pre-flush UserProfile object for shylock when loading /stats,
which ultimately would result in /stats appearing totally broken.
2018-07-09 15:13:59 +05:30
Eeshan Garg 0a43e5e257 Replace all user-facing references to "invite-only" with "private".
Fixes #9611.
2018-06-12 13:37:45 -04:00
Nikhil Kumar Mishra fa9d79e203 stats: Add 1 day actives and total users to number of users chart. 2018-05-20 10:56:16 -07:00
Rishi Gupta 66a589c7a7 stats: Extend get_chart_data to support charts with multiple CountStats. 2018-05-20 10:56:16 -07:00
Rishi Gupta 08bf0a66b8 stats: Refactor the get_time_series_by_subgroup calls in get_chart_data.
This code is going to end up pretty complex -- each stat has multiple levels
of aggregation (UserCount, RealmCount, InstallationCount), and refinement
(subgroups), and soon we'll have charts that take data from multiple stats
as input.

Not sure what the best way to present it is, but hopefully this simplifies
it a bit.
2018-05-20 10:56:16 -07:00
Nikhil Kumar Mishra 26decb4c48 stats: Add 1day_actives::day CountStat to analytics tables. 2018-05-20 10:56:16 -07:00
Rishi Gupta 1af7fc7344 stats: Add /stats/installation. 2018-05-18 15:12:36 -07:00
Rishi Gupta 2fe3fba6ce stats: Rename data.realm to data.everyone.
We use "Everyone" for the button labels already.

Soon we'll support "Everyone" meaning either the installation or the realm,
depending on the URL route used to access the stats.
2018-05-18 15:12:36 -07:00
Rishi Gupta af758755bd stats: Rename target_realm_name to target_name in stats.html. 2018-05-18 15:12:36 -07:00
Rishi Gupta e099959a41 stats: Move API route computation from frontend to backend.
Will make it easier to extend to additional routes.
No changes in behavior.
2018-05-18 15:12:36 -07:00
Tim Abbott f0ef335412 models: Remove unused ModelReprMixin class.
It appeared to be used as a base class in various Django migrations,
but because it didn't define any model fields, it wasn't actually.
2018-05-15 19:11:22 -07:00
Yago González 184bd8304e i18n: Tag missing strings for translation. 2018-05-12 16:44:56 -07:00
Aditya Bansal 5adf983c3c analytics: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
Shubham Dhama 03a2a9c792 activity: Add realm stats link to "realm activtiy table". 2018-04-18 11:07:00 -07:00
Shubham Dhama b26c38bc47 analytics: Make stats of all realms accessible to server admins.
In this commit:
Two new URLs are added, to make all realms accessible for server
admins. One is for the stats page itself and another for getting
chart data i.e. chart data API requests.
For the above two new URLs corresponding two view functions are
added.
2018-04-18 11:06:50 -07:00
neiljp (Neil Pilgrim) 8b697b4093 mypy: Annotate stream_data in populate_analytics_db.py handle function. 2018-03-25 08:59:08 -07:00
neiljp (Neil Pilgrim) 9e1dbde82d mypy: Final small migrations to python3.5 annotations in many files. 2018-03-12 11:23:30 -07:00
Archana BS b5a860b234 analytics: Populate messages_in_stream:is_bot:day in dev. 2018-03-04 13:17:00 -08:00
Tim Abbott 6bbce451bc analytics: Fix a minor mypy error. 2018-02-12 11:57:47 -08:00
Aman Jain a0b58b1560 activity: Add a link to copy list of realm admins.
Tweaked by tabbott to do the database queries properly.

This should help user to copy realm admin emails in a go.
Fixes: #7885
2018-02-12 08:55:45 -08:00
Rishi Gupta 1d581a9c6e nagios: Add nagios check for analytics state.
This should help us detect issues where the analytics cron jobs aren't
running properly.

The cron/nagios part of the implementation done by tabbott.
2018-02-09 16:36:05 -08:00
rht 9a8d2244ca django-2.0: Shift to resolvers from urlresolvers.
The old name is deprecated.
2018-01-30 10:53:54 -08:00
rht 8106a25e61 django-2.0: Add on_delete on ForeignKeys.
In Django 2.0, one must specify the on_delete behavior for all
ForeignKeys explicitly.
2018-01-30 10:53:54 -08:00
Greg Price b830b446f1 logging: Reduce `create_logger` to new `log_to_file`.
The name `create_logger` suggests something much bigger than what this
function actually does -- the logger doesn't any more or less exist
after the function is called than before.  Its one real function is to
send logs to a specific file.

So, pull out that logic to an appropriately-named function just for
it.  We already use `logging.getLogger` in a number of places to
simply get a logger by name, and the old `create_logger` callsites can
do the same.
2017-12-12 17:17:08 -08:00
Greg Price ebcf0b4876 logging: Stop having `create_logger` force loglevels to INFO.
This is already the loglevel we set on the root logger, so this has no
effect -- except in tests, where `test_settings.py` attempts to set
some of these same loggers to higher loglevels.  Because the
`create_logger` call generally runs after we've configured settings,
it clobbers that effect.

The code in `test_settings.py` that tries to suppress logs only works
because it also sets `propagate=False`, which has nothing to do with
loglevels but does cause logs at this logger (and descendants) to be
dropped completely unless we've configured handlers for this logger
(or one of its relevant descendants.)
2017-12-12 17:17:07 -08:00
Greg Price 20b2c11830 activity: Show the time the data is from.
I've wanted this when looking at a tab from the day before.
Also provides the date and time in UTC, which is handy for
interpreting some of the data.

Pretty sure this is not the world's cleanest way to do this in the
front-end code.  It'll do for now.
2017-12-12 15:30:03 -08:00
Greg Price bdea0960de activity: Supply a missing field.
In prod this has no effect, but in dev we render a warning here,
which makes the table not look right.  Fix it.
2017-12-12 15:30:03 -08:00