Commit Graph

645 Commits

Author SHA1 Message Date
Lauryn Menard 3be622ffa7 backend: Add request as parameter to json_success.
Adds request as a parameter to json_success as a refactor towards
making `ignored_parameters_unsupported` functionality available
for all API endpoints.

Also, removes any data parameters that are an empty dict or
a dict with the generic success response values.
2022-02-04 15:16:56 -08:00
Anders Kaseorg 78e54a0d7a python: Replace deprecated jinja2.utils.Markup with markupsafe.Markup.
Fixes “DeprecationWarning: 'jinja2.Markup' is deprecated and will be
removed in Jinja 3.1. Import 'markupsafe.Markup' instead.”

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:22:48 -08:00
Anders Kaseorg 1696144df7 docs: Consistently hyphenate “self-host” and “self-service”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-05 16:21:35 -08:00
Eeshan Garg 2cdaae681d actions: Rename do_change_plan_type -> do change_realm_plan_type.
We will soon be adding an equivalent function for RemoteZulipServer,
so it makes sense to rename this function to be more descriptive.
2021-12-06 16:18:53 -08:00
Eeshan Garg b325a4f1be realm: Rename plan type constants to be more descriptive.
It is confusing to have the plan type constants not be namespaced
by the thing they represent. We already have a namespacing
convention in place for constants, so we should use it for
Realm.plan_type as well.
2021-10-19 12:20:39 -07:00
Vishnu KS aff52722a7 billing: Add PLUS as a plan_type to Realm. 2021-10-15 17:27:50 -07:00
Eeshan Garg 1dec97c925 analytics: Use user IDs to get user activity summaries.
Using user IDs instead of emails is more reliable since users can
have arbitrarily complex emails that are hard to encode in a URL.
This has led to NoReverseMatch exceptions in the past.
2021-10-13 15:30:27 -07:00
Eeshan Garg f25230c7d4 analytics/support: Add ability to edit realm org type. 2021-10-07 14:08:11 -07:00
Mateusz Mandera 3205f680c1 do_invite_users: Turn some args into kwargs-only. 2021-09-10 16:53:03 -07:00
shanukun 8c1ea78d7d invite: Extend invite api for handling expiration duration.
This extends the invite api endpoints to handle an extra
argument, expiration duration, which states the number of
days before the invitation link expires.

For prereg users, expiration info is attached to event
object to pass it to invite queue processor in order to
create and send confirmation link.
In case of multiuse invites, confirmation links are
created directly inside do_create_multiuse_invite_link(),

For filtering valid user invites, expiration info stored in
Confirmation object is used, which is accessed by a prereg
user using reverse generic relations.

Fixes #16359.
2021-09-10 16:53:03 -07:00
PIG208 7386918539 typing: Use accurate type hints for dictionaries.
This fixes the mypy errors related to dictionaries with django-stubs.
2021-08-20 06:02:28 -07:00
PIG208 c99c423c81 support: Remove `is True` in comparisons.
This is a followup for #19432 that fixes a remaining issue of the
original PR.
2021-08-10 12:08:13 -07:00
Anders Kaseorg 09564e95ac mypy: Add types-psycopg2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
PIG208 94685e1afb analytics: Refactor the support view to use REQ. 2021-08-08 17:11:18 -07:00
Tim Abbott 65557c458d stats: Avoid referencing remote tables outside conditional. 2021-08-03 11:15:39 -07:00
PIG208 ec99c1b58d analytics: Correctly type aggregate_table with Union.
Since mypy doesn't accept redefinition of the same variable within the
same scope, we need to use type annotations with Union to correctly
type aggregate_table. Note that the type cast is necessary for mypy to
narrow the type of aggregate_table.
2021-07-27 12:17:01 -07:00
PIG208 6ab42b7663 analytics: Use type alias for `fixture_data`.
This saves us from repeating the lengthy type annotation for
`fixture_data`.
2021-07-27 12:17:01 -07:00
PIG208 e517f967b8 typing: Amend inaccurate type annotations.
This amend some type annotations that turn out to be inaccurate with
django-stubs.
2021-07-27 12:17:00 -07:00
PIG208 caaa424ef5 typing: Use assertions for broader types.
For types like `Union[Realm, UserProfile, Stream]` and
`Union[AnonymousUser, AbstractBaseUser]`, we need assertions to
tell mypy which type we would be expecting.
2021-07-27 11:44:54 -07:00
PIG208 8121d2d58d typing: Fix misuse of HttpResponse.
Amend usage of HttpResponse when appropriate.
2021-07-27 14:31:19 +08:00
PIG208 d2af20eb2a typing: Do explicit type conversion when appropriate.
When calling some functions or assigning values to certain attributes,
the arguments/right operand do not match the exact type that the
functions/attributes expect, and thus we fix that by converting types
beforehand.
2021-07-26 15:09:07 -07:00
PIG208 cd678232bd analytics: Extract request.user for typecheck. 2021-07-26 14:48:45 -07:00
PIG208 7d1c475f69 typing: Use assertions for function arguments.
Utilize the assert_is_not_None helper to eliminate errors of
'Argument x to "Foo" has incompatible type "Optional[Bar]"...'
2021-07-26 14:48:45 -07:00
PIG208 8a91d1c2b1 analytics: Fix type annotation for keys.
realm_data, user_data, stream_data and installation_data allow the key to
be int, str, or None.
This fix the type annotation for these dictionaries.
2021-07-26 14:48:45 -07:00
PIG208 df1bf9e352 analytics: Fix type annotation for sql_data_collector. 2021-07-26 14:46:45 -07:00
PIG208 66b1a4e7ca backend: Add None-checks with assertions and if-elses.
This fixes a batch of mypy errors of the following format:
'Item "None" of "Optional[Something]" has no attribute "abc"'
2021-07-24 15:00:21 -07:00
PIG208 495a8476be tests: Use assertion to enforce None-checks in tests.
This fixes a batch of mypy errors of the following format:
'Item "None" of "Optional[Something]" has no attribute "abc"

Since we have already been recklessly using these attritbutes
in the tests, adding assertions beforehand is justified presuming
that they oughtn't to be None.
2021-07-24 09:54:21 -07:00
Anders Kaseorg fb3ddf50d4 python: Fix mypy no_implicit_reexport errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-16 14:02:31 -07:00
Tim Abbott 4d0611c363 support: Add test for get_org_type_display_name.
This apparently lost coverage while I was integrating
faa695e86d.
2021-07-15 11:11:23 -07:00
Eeshan Garg 366b7fe2f5 analytics: Display org_type in /activity/support results. 2021-07-15 10:14:39 -07:00
Eeshan Garg 434c262365 analytics: Display org_type on /activity.
Moving forward we are hoping to collect data on org types from our
users, so it makes sense to display the org type on the "Counts"
tab of our /activity page.
2021-07-15 10:14:38 -07:00
akshatdalton e203112fd4 refactor: Use `assert_length` helper instead of `assertTrue/assertEqual`. 2021-07-13 13:03:38 -07:00
Tim Abbott c804ab27d5 actions: Rename do_activate_user to do_activate_mirror_dummy_user.
This function had a confusing name, which could result in someone
using it unintentionally when they meant do_reactivate_user.

We also add docstrings for both functions.
2021-07-08 17:33:17 -07:00
PIG208 dcbb2a78ca python: Migrate most json_error => JsonableError.
JsonableError has two major benefits over json_error:
* It can be raised from anywhere in the codebase, rather than
  being a return value, which is much more convenient for refactoring,
  as one doesn't potentially need to change error handling style when
  extracting a bit of view code to a function.
* It is guaranteed to contain the `code` property, which is helpful
  for API consistency.

Various stragglers are not updated because JsonableError requires
subclassing in order to specify custom data or HTTP status codes.
2021-06-30 16:22:38 -07:00
Tim Abbott e0c7581168 analytics: Rename get_activity to get_installation_activity. 2021-06-17 17:39:28 -07:00
Tim Abbott 0e358e4cc2 analytics: Rename analytics/views/legacy.py. 2021-06-17 17:39:28 -07:00
Tim Abbott e083134764 analytics: Extract analytics/views/user_activity.py. 2021-06-17 17:39:28 -07:00
Tim Abbott 0d2e5cc390 analytics: Extract analytics/views/realm_activity.py. 2021-06-17 17:39:28 -07:00
Tim Abbott 7099dff91a analytics: Extract common /activity views functions.
This will be helpful for further splitting the giant files here.
2021-06-17 17:39:28 -07:00
Tim Abbott d22acb23bc analytics: Extract analytics/views/support.py. 2021-06-17 17:39:28 -07:00
Tim Abbott 5d8613a02c analytics: Extract analytics/views/stats.py.
This is much higher quality code than some of the older /activity
code, and it's nice to give it its own home.
2021-06-17 17:39:28 -07:00
Tim Abbott ee54ecfccf analytics: Split test_views.py.
This will aligns the test organization with an upcoming reorganization
of the code itself.
2021-06-17 17:39:28 -07:00
Tim Abbott 5891849b0d analytics: Extract test_stats_views.
This will help make the analytics views codebase more readable.
2021-06-17 17:39:28 -07:00
Tim Abbott 9c8ddef5f2 analytics: Create analytics/views/ directory. 2021-06-17 17:39:28 -07:00
Tim Abbott c36b55e418 analytics: Remove stream_stats management command.
We will eventually provide features like this in the /stats UI, and in
any case this doesn't work and hasn't been maintained in years.
2021-06-17 11:30:42 -07:00
Vishnu KS 4e26746b98 tests: Move tests of /activity pages from zerver to analytics module.
This module deals with the testing of /activity, /realm_activity
and /user_activity. All these pages reside in analytics module.
Keeping these tests in zerver/tests is kind is not appropriate
since person who makes changes to /activity pages would not think
it is necessary to run tests in zerver. So better to keep them
in the analytics module.
2021-06-11 07:41:02 -07:00
Vishnu KS d32de76c23 activity: Rename amount variables to arr. 2021-06-11 07:41:02 -07:00
Vishnu KS cfbea80f08 activity: Don't show plan, ARR and discount when billing is disabled. 2021-06-11 07:41:02 -07:00
Vishnu KS 40ab415005 activity: Show effective rate of realms in /activity page. 2021-06-11 07:41:02 -07:00
Vishnu KS 8c055107d9 exceptions: Move InvitationError to zerver/lib/exceptions. 2021-06-09 17:42:38 -07:00
Abhijeet Prasad Bodas 352634a851 tests: Consistently use assert_length helper.
This helper does some nice things like printing out
the data structure incase of failure.
2021-05-19 11:55:56 -07:00
Anders Kaseorg dd3fa4ac52 templates: Mark all void tags as self-closing.
This reverses the policy that was set, but incompletely enforced, by
commit 951514dd7d.  The self-closing tag
syntax is clearer, more consistent, simpler to parse, compatible with
XML, preferred by Prettier, and (most importantly now) required by
FormatJS.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-21 09:49:34 -07:00
Vishnu KS ab771e4b19 support: Show the first human user in realm search result. 2021-04-16 13:22:02 -07:00
Vishnu KS 8139896e3d support: Show realm owners in realm search results. 2021-04-16 13:18:46 -07:00
Vishnu KS 97765798d7 support: Rename realm_admin_emails to get_realm_admin_emails_as_string. 2021-04-16 13:18:46 -07:00
Anders Kaseorg e7ed907cf6 python: Convert deprecated Django ugettext alias to gettext.
django.utils.translation.ugettext is a deprecated alias of
django.utils.translation.gettext as of Django 3.0, and will be removed
in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Tim Abbott 6b1ca15ba7 activity: Use realm owners, not realm administrators.
The analytics system was not properly updated when we replaced realm
owners with realm administrators.
2021-04-08 17:47:23 -07:00
Siddharth Asthana 47e478945b support: Create RealmAuditLog when updating realm_subdomain. 2021-04-03 08:01:46 -07:00
Siddharth Asthana 80c9243c6a support: Create RealmAuditLog when updating sponsorship status. 2021-04-03 08:01:46 -07:00
Siddharth Asthana ddbc6d7662 support: Create RealmAuditLog when realm reactivation email is sent. 2021-04-03 08:01:46 -07:00
Siddharth Asthana 233c4d520c support: Create RealmAuditLog when updating billing_method.
This commit also makes acting_user as a mandantory argument and fixes
the tests accordingly.
2021-04-03 08:01:46 -07:00
Siddharth Asthana 6945ed3587 support: Pass acting_user to attach_discount_to_realm. 2021-04-03 08:01:46 -07:00
Siddharth Asthana c3f37c2a64 support: Pass acting_user to do_change_plan_type.
acting_user is now a mandatory field, so wherever this function is used,
we are passing acting_user as well.
2021-04-03 08:01:46 -07:00
shanukun 88262a484c support: Fix arguments of timesince for expires_in.
`expires_in` (remaining time before the invite expires) should
be calculated from the time at present, not from the time when
confirmation link was sent.
2021-04-02 22:01:57 -07:00
shanukun 4b00e5da72 refactor: Make acting_user a mandatory kwarg for do_deactivate_realm. 2021-04-02 14:44:41 -07:00
Mateusz Mandera f329878376 migrations: Subscription.is_user_active denormalization - step one.
This adds the is_user_active with the appropriate code for setting the
value correctly in the future. In the following commit a migration to
backfill the value for existing Subscriptions will be added.

To ensure correct user_profile.is_active handling also in tests, we
replace all direct .is_active mutation with calls to appropriate
functions.
2021-03-30 09:19:03 -07:00
shanukun c95061e9b9 refactor: Make acting_user a mandatory kwarg for do_deactivate_user. 2021-03-29 15:51:45 -07:00
shanukun 8f3ae715c0 refactor: Make acting_user a mandatory kwarg for do_reactivate_user. 2021-03-29 15:51:45 -07:00
shanukun 459710a897 refactor: Make acting_user a mandatory kwarg for do_set_realm_property. 2021-03-29 15:51:45 -07:00
Tim Abbott 4bdaec7a8d analytics: Stop overriding debug_mode.
This had the same value as the default in default_page_params, and
appears to have been present since this page was introduced in
b26c38bc47.
2021-03-26 09:41:12 -07:00
Vishnu KS 1a126f8bf2 tests: Don't harcode email addresses in analytics.tests.test_views. 2021-03-22 18:10:03 -07:00
Vishnu KS 79c4a25d20 tests: Create assert_user_details_in_html_response helper function. 2021-03-22 18:10:03 -07:00
Vishnu KS a404711740 support: Show the actual user email in /support. 2021-03-22 18:10:03 -07:00
Vishnu KS 38e7903425 support: Show role of the user in search result. 2021-03-22 18:10:03 -07:00
aryanshridhar 04fb0552a6 analytics: escape HTML correctly when generating links.
Wrapped the html text within html.escape function to convert
special characters into HTML-safe string while generating link.
2021-03-18 15:08:31 -07:00
Tim Abbott 0da1bd43e9 analytics: Remove buggy HttpResponseNotFound text.
Had this been in normal route, this would have been an XSS bug, as we
were passing what the developer clearly believed to be plain text into
an HTML 404 page.

The affected routes have @require_server_admin, a permission that we
do not expect any self-hosted users to have ever enabled (as it is
undocumented and doing so is only possible manually via a `manage.py
shell`, and we believe to only be useful for running a SaaS service
like zulip.com).  So the security impact is limited to a handful of
staff of zulip.com and this isn't a candidate for a CVE.

Thanks to GitHub's CodeQL for finding this.
2021-03-18 12:16:15 -07:00
Mateusz Mandera d91d3a05b9 tests: Use do_create_realm where possible.
Using do_create_realm should be preferred over manual creation where
possible, as it creates more realistic data.
2021-03-14 08:50:02 -07:00
shanukun fafe1a31d7 refactor: Make acting_user a mandatory kwarg for do_activate_user. 2021-02-25 17:58:00 -08:00
shanukun 4b67946605 refactor: Make acting_user a mandatory kwarg for do_create_user. 2021-02-25 17:58:00 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 5028c081cb python: Merge concatenated string literals that Black would uglify.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 9773c0f1a8 python: Fix string literal concatenation mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:02:51 -05:00
Siddharth Asthana 6c888977a6 change_subdomain: Create a deactivated realm on updating subdomain.
When changing the subdomain of a realm, create a deactivated realm with
the old subdomain of the realm, and set its deactivated_redirect to the
new subdomain.
Doing this will help us to do the following:
- When a user visits the old subdomain of a realm, we can tell the user
that the realm has been moved.
- During the registration process, we can assure that the old subdomain
of the realm is not used to create a new realm.

If the subdomain is changed multiple times, the deactivated_redirect
fields of all the deactivated realms are updated to point to the new
uri.
2021-01-07 14:15:22 -08:00
Vishnu KS 9fe39646fa analytics: Specify exact end_time in realm summary query.
Fetchings rows with end_time within the last 25 hours would result
in the realmcount queries returning two rows for each realm
if the analytics page was opened within an hour since the
count stats were updated.
2020-12-22 16:44:31 -08:00
Vishnu KS 9d5a1271d4 analytics: Make last_successful_fill handle FillState.STARTED case properly.
Subtracting an hour from end_time is correct only for CountStats with
hourly frequency. For daily frequency we should subtract a day instead.
2020-12-22 16:44:31 -08:00
Vishnu KS 235a347639 analytics: Move last_successful_fill to CountStat.
This is a prep commit. Currenty we only pass CountStat.property
to last_successful_fill function. But it needs access to
CountStat.time_increment as well. We can pass the entire CountStat
object to the function as a workaround. But making last_successful_fill
a property of CountStat seems to be much more cleaner.
2020-12-22 16:44:31 -08:00
Vishnu KS 189e9a2759 analytics: Create time_increment property in CountStat. 2020-12-22 16:44:31 -08:00
Vishnu KS 7b2f16bc5c support: Mention % in attach discount success message. 2020-12-17 17:09:20 -08:00
Vishnu KS dd5bcb97e8 support: Allow user look up through full name search. 2020-12-17 17:07:28 -08:00
Siddharth Asthana 6f962c1815 support: Add ability to change subdomain of realms. 2020-12-06 00:48:10 -08:00
Siddharth Asthana 37e158b9ba support: Rename message -> success_message. 2020-12-06 00:48:10 -08:00
m-e-l-u-h-a-n cbfd6464a5 logging: replace mock.patch() for logging with assertLogs()
This commit removes mock.patch with assertLogs().

* Adds return value to do_rest_call() in outgoing_webhook.py, to
  support asserting log output in test_outgoing_webhook_system.py.

* Logs are not asserted in test_realm.py because it would require to users
  to be queried using users=User.objects.filter(realm=realm) and the order
  of resulting queryset varies for each run.

* In test_decorators.py, replacement of mock.patch is not done because
  I'm not sure if it's worth the effort to replace it as it's a return
  value of a function.

Tweaked by tabbott to set proper mypy types.
2020-10-29 15:37:45 -07:00
Vishnu KS f1ceab69c9 activity: Speed up the main query in /activity. 2020-10-23 12:45:20 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Steve Howell 31eb97ddde performance: Fix do_mark_stream_messages_as_read.
This function no longer asks for data that it
doesn't need.
2020-10-16 12:58:11 -07:00
Anders Kaseorg f0ae2a88f2 activity: Show server time rather than browser time.
This gets rid of an inline <script> that was blocking #16451.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-08 19:17:35 -07:00
Abhijeet Prasad Bodas bd61b570aa stats: Fix strings passed to _() not translated.
Part of #16094.
Strings constructed by _() were not being
translated in the /stats page.
This was because session variable was not set.
Ideally this should have been a part of b82bda9.
2020-10-02 14:56:20 -07:00
Vishnu KS 45b7673f4a support: Show realm string_id instead of name in alerts.
Since realm.name can be empty.
2020-09-28 15:37:49 -07:00
Vishnu KS 510efbc1a8 support: Add option to change billing method. 2020-09-28 15:37:49 -07:00