Commit Graph

665 Commits

Author SHA1 Message Date
Alex Vandiver 330141f55d invites: Switch new LIMITED-plan heuristic to enforcing. 2023-03-16 11:41:49 -07:00
Anders Kaseorg 2d9b2a2a05 models: Remove type prefixes from __str__ values.
The Django convention is for __repr__ to include the type and __str__
to omit it.  In fact its default __repr__ implementation for models
automatically adds a type prefix to __str__, which has resulted in the
type being duplicated:

    >>> UserProfile.objects.first()
    <UserProfile: <UserProfile: emailgateway@zulip.com <Realm: zulipinternal 1>>>

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-08 22:56:55 -08:00
Alex Vandiver 36da7783ce invites: _max_invites is currently never None.
dc1eeef30a made the column nullable, with the meaning for null of
"use the current `settings.INVITES_DEFAULT_REALM_DAILY_MAX`."
However, 8a95526ced switched to calling `do_change_plan_type` during
realm creation, which sets `realm.max_invites` based on the plan type,
thus ensuring that no new realms have their `_max_invites` set to
null.

Check `max_invites` instead of `_max_invites`.  This requires test
adjustments for the fact that `apply_invite_realm_heuristics` is now
run.
2023-03-07 15:04:39 -08:00
Anders Kaseorg d3efd4c095 python: Import F, Q, QuerySet from their canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg 0628c3cac8 migrations: Import BaseDatabaseSchemaEditor from its canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Alex Vandiver 04e7621668 upload: Rename upload_message_image_from_request.
The table is named Attachment, and not all of them are images.
2023-03-02 16:36:19 -08:00
Sahil Batra 3950a8e19d test_helpers: Rename reset_emails_in_zulip_realm.
This commit renames reset_emails_in_zulip_realm function to
reset_email_visibility_to_everyone_in_zulip_realm which makes
it more clear to understand what the function actually does.

This commit also adds a comment explaining what this function
does.
2023-03-01 12:17:11 -08:00
Lauryn Menard a0fd7b2afc private-messages: Update translated backend strings use "direct message".
Updates user-facing translated strings containing "private message" on
the backend to use "direct message" instead.
2023-02-24 11:47:26 -08:00
Sahil Batra 0ed5f76063 settings: Add backend code for using user email_address_visibility setting.
This commits update the code to use user-level email_address_visibility
setting instead of realm-level to set or update the value of UserProfile.email
field and to send the emails to clients.

Major changes are -

- UserProfile.email field is set while creating the user according to
RealmUserDefault.email_address_visbility.

- UserProfile.email field is updated according to change in the setting.

- 'email_address_visibility' is added to person objects in user add event
and in avatar change event.

- client_gravatar can be different for different users when computing
avatar_url for messages and user objects since email available to clients
is dependent on user-level setting.

- For bots, email_address_visibility is set to EVERYONE while creating
them irrespective of realm-default value.

- Test changes are basically setting user-level setting instead of realm
setting and modifying the checks accordingly.
2023-02-10 17:35:49 -08:00
Anders Kaseorg df001db1a9 black: Reformat with Black 23.
Black 23 enforces some slightly more specific rules about empty line
counts and redundant parenthesis removal, but the result is still
compatible with Black 22.

(This does not actually upgrade our Python environment to Black 23
yet.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-02 10:40:13 -08:00
Anders Kaseorg ff1971f5ad ruff: Fix SIM105 Use `contextlib.suppress` instead of try-except-pass.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg b0e569f07c ruff: Fix SIM102 nested `if` statements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg 3025d9a63a ruff: Fix Q002 Single quote docstring found.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Alex Vandiver 0e10ccc137 analytics: Pass the right value for realm. 2022-12-08 12:42:41 -08:00
Lauryn Menard b686e6ce9e analytics-realm-activity: Add link to realm's stats page.
Adds the  pie chart icon stats page link to the realm_activity
page header.
2022-12-02 17:30:30 -08:00
Lauryn Menard c05c25e742 analytics-activity: Fix UTC timezone and adjust header type.
Fixes an error with how the `utctime` was being generated and makes
the header on the summary table the same type as the other tabs.
2022-12-02 17:30:30 -08:00
Lauryn Menard c75e5c8ba6 analytics-activity: Add links for realm's Zulip and support page. 2022-12-02 17:30:30 -08:00
Lauryn Menard 90d0531e20 analytics-activity: Remove link with realm owners emails.
Note that a link to copy these emails to the clipboard is still
availabe on the realm's support page.
2022-12-02 17:30:30 -08:00
Anders Kaseorg 73c4da7974 ruff: Fix N818 exception name should be named with an Error suffix.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-17 16:52:00 -08:00
Anders Kaseorg 924d530292 ruff: Fix N813 camelcase imported as lowercase.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 09:29:11 -08:00
Lauryn Menard a3f6220fe4 analytics: Add summary statistic for guest users in realm.
Adds the count of users with the role of guest to the stats view
`page_params` via a database query. This information is then added
to the summary statistics section of the analytics page after being
formatted by `stats.js`.

Creates Bassanio as a guest user in the database for the analytics
realm.

Fixes #20162.
2022-10-17 11:53:59 -07:00
Zixuan James Li 46329a2710 test_classes: Create a dedicate helper for query count check.
This adds a helper based on testing patterns of using the "queries_captured"
context manager with "assert_length" to check the number of queries
executed for preventing performance regression.

It explains the rationale of checking the query count through an
"AssertionError" and prints the queries captured as assert_length does,
but with a format optimized for displaying the queries in a more
readable manner.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-17 11:32:52 -07:00
Mateusz Mandera 00b3546c9f models: Add denormalized .realm column to Message.
This commit adds the OPTIONAL .realm attribute to Message
(and ArchivedMessage), with the server changes for making new Messages
have this set. Old Messages still have to be migrated to backfill this,
before it can be non-nullable.

Appropriate test changes to correctly set .realm for Messages the tests
manually create are included here as well.
2022-10-07 10:09:38 -07:00
Lauryn Menard df7a1cec93 analytics: Add summary statistic for upload/storage space in use.
Adds the realm's used storage space for attachments to the stats
view `page_params`. This information is then added to the summary
statistics section of the analytics page after being formatted by
`stats.js`.

Uses the emoji test image to create an `Attachment` in the database
for the analytics realm. Even though it doesn't create a message
to claim the attachment, it still is sent as storage space used
data for the analytics `/stats/` page.
2022-10-07 10:08:09 -07:00
Zixuan James Li 4c3c976174 models: Implicitly type model fields with django-stubs.
Previously, we type the model fields with explicit type annotations
manually with the approximate types. This was because the lack of types
for Django.

django-stubs provides more specific types for all these fields that
incompatible with our previous approximate annotations. So now we can
remove the inline type annotations and rely on the types defined in the
stubs. This allows mypy to infer the types of the model fields for us.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-05 16:15:56 -07:00
Sahil Batra b0de5c0f36 streams: Set can_remove_subscribers_group while creating streams.
This commit sets can_remove_subscribers_group to admins system
group while creating streams as it will be the default value
of this setting. In further we would provide an option to set
value of this setting to any user group while creating streams
using API or UI.
2022-09-14 16:03:11 -07:00
Anders Kaseorg 868e130b5f populate_analytics_db: Remove unnecessary mock.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-14 09:36:56 -07:00
Zixuan James Li ca0d2f6854 decorator: Refactor decorators expecting UserProfile with ParamSpec.
Decorators like `require_server_admin_api` turns user_profile into a
positional-only parameter, requiring the callers to stop passing it as a
keyword argument.

Functions like `get_chart_data` that gets decorated by both
`require_non_guest_user` and `has_request_variables` now have accurate
type annotation during type checking, with the first two parameters
turned into positional-only, and thus the change in
`analytics.views.stats`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-06 16:19:48 -07:00
Zixuan James Li adae8b6d42 request: Refactor has_request_variables with ParamSpec.
This makes `has_request_variables` more generic, in the sense of the return
value, and also makes it more accurate, in the sense of requiring the
first parameter of the decorated function to be `HttpRequest`, and
preserving the function signature without using `cast`.

This affects some callers of `has_request_variables` or the callers of its
decoratedfunctions in the following manners:

- Decorated non-view functions called directly in other functions cannot
use `request` as a keyword argument. Becasue `Concatenate` turns the
concatenated parameters (`request: HttpRequest` in this case) into
positional-only parameters. Callers of `get_chart_data` are thus
refactored.

- Functions to be decorated that accept variadic keyword arguments must
define `request: HttpRequest` as positional-only. Mypy in strict mode
rejects such functions otherwise because it is possible for the caller to
pass a keyword argument that has the same name as `request` for `**kwargs`.
No defining `request: HttpRequest` as positional-only breaks type safety
because function with positional-or-keyword parameters cannot be considered
a subtype of a function with the same parameters in which some of them are
positional-only.

Consider `f(x: int, /, **kwargs: object) -> int` and `g(x: int,
**kwargs: object) -> int`. `f(12, x="asd")` is valid but `g(12, x="asd")`
is not.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-06 16:19:48 -07:00
Zixuan James Li 95394de186 decorator: Refactor require_server_admin_api with ParamSpec.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-30 18:15:33 -07:00
Anders Kaseorg 73374996a5 analytics: Add Composable type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-30 06:46:34 -07:00
Mateusz Mandera 30c26b9510 support: Correctly show Unspecified org type at /activity/support.
Because the org type is marked as "hidden" the HTML was being generated
for orgs with Unspecified .org_type with no <option> selected, meaning
it was displayed on the page using the first <option> in the list
(Business). The /support endpoint should ignore the "hidden" property,
since there's no reason not to - we only want to hide this org type from
regular users during Org registration.
2022-07-29 14:56:34 -07:00
Mateusz Mandera cf74d7d140 realm_reactivation: Prevent realm reactivation link reuse.
This uses the approach analogical to EmailChangeStatus for email change
confirmation links.
2022-07-26 17:14:26 -07:00
Mateusz Mandera fa7700df11 confirmation: Rename STATUS_ACTIVE to STATUS_USED.
That's much more descriptive of what that value actually means about the
Confirmation objects.
2022-07-21 15:17:37 -07:00
Mateusz Mandera 0ffdc96301 support: Adjust "has been clicked" phrasing about confirmation links.
The "clicked" phrasing is not accurate, because e.g. if a user did click
their invitation link but didn't submit the registration form, the
support page will still claim about the link "has never been clicked".

"Used" is a better general phrase. If we want to track whether links
have been specifically *clicked*, we'll need to implement that
separately.
2022-07-21 15:17:37 -07:00
Anders Kaseorg 81892df176 requirements: Upgrade to Django 4.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-13 16:07:17 -07:00
Anders Kaseorg 59174694af support: Skip corporate import unless BILLING_ENABLED.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-13 15:24:10 -07:00
Zixuan James Li a76c7d374b support: Display plan data for users and confirmations.
Users and confirmation objects with the type
`Confirmation.USER_REGISTRATION` or `Confirmation.INVITATION` may have
plan data associated with them but not displayed previously due to a
bug.

This fixes this issue and adds test cases to verify that the realm
details correctly displays the plan data.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 15:03:26 -07:00
Zixuan James Li e836aa31ad test_support_views: Extract lear_realm.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 15:03:26 -07:00
Zixuan James Li aa5d540845 test_support_views: Add create_invitation helper.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 15:03:26 -07:00
Zixuan James Li c727f1f65d test_support_views: Add get_check_query_result helper.
This is a prep commit for a refactoring that fixes an issue with plan
data not being displayed when the realm is displayed by the query result
of users or confirmation objects.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 15:03:26 -07:00
Zixuan James Li 0d93257111 support: Extract PlanData as a dataclass.
This avoids monkey-patching `CustomerPlan` and other related information
onto the `Realm` object by having a separate dictionary with the realm
id as the key, each corresponds to a `PlandData` dataclass.

This is a part of the django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 15:03:26 -07:00
Zixuan James Li eeaeb5a821 confirmation: Refactor get_confirmations to accept confirmation ids.
Two of the callers of `get_confirmations` uses a `QuerySet` of confirmation
objects instead of their ids to filter the confirmations. This refactors
`get_confirmations` so that it is typed to accept `Iterable[int]` that
is a list of ids.

It's worth noting that this might be less performant than the previous
approach since it requires more queries when we force the ids into lists
without having django creating a nested query. But the performance
is not a concern here compared to clarity.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-13 14:46:51 -07:00
Zixuan James Li 0528935b82 stats: Tighten function signatures with generic QuerySet.
Luckily `QuerySet` supports type variables. This allows us
to type table_filtered_to_id more accurately.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-07 11:28:06 -07:00
Zixuan James Li 81ab0b3615 user_activity: Tighten function signatures with generic QuerySet.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-07 11:27:43 -07:00
Zixuan James Li 32ad110af2 analytics: Tighten function signatures with generic QuerySet.
Note that the `list` conversion before assignment to `all_records`
is not necessary for its usage in `realm_user_summary_table` from
a typing perspective.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-07 11:27:43 -07:00
Zixuan James Li ab1bbdda65 typing: Broaden type annotations for QuerySet compatibility.
To explain the rationale of this change, for example, there is
`get_user_activity_summary` which accepts either a `Collection[UserActivity]`,
where `QuerySet[T]` is not strictly `Sequence[T]` because its slicing behavior
is different from the `Protocol`, making `Collection` necessary.

Similarily, we should have `Iterable[T]` instead of `List[T]` so that
`QuerySet[T]` will also be an acceptable subtype, or `Sequence[T]` when we
also expect it to be indexed.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-07 11:27:42 -07:00
Zixuan James Li fd9a0f4274 typing: Apply trivial none-checks with assertions as necessary.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li 426f8ce385 tests: Replace `HttpResponse` with `TestHttpResponse.`
Since `HttpResponse` is an inaccurate representation of the
monkey-patched response object returned by the Django test client, we
replace it with `_MonkeyPatchedWSGIResponse` as `TestHttpResponse`.

This replaces `HttpResponse` in zerver/tests, analytics/tests, coporate/tests,
zerver/lib/test_classes.py, and zerver/lib/test_helpers.py with
`TestHttpResponse`. Several files in zerver/tests are excluded
from this substitution.

This commit is auto-generated by a script, with manual adjustments on certain
files squashed into it.

This is a part of the django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-08 11:25:03 -07:00
Zixuan James Li a142fbff85 tests: Refactor away result.json() calls with helpers.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-06 23:06:00 -07:00