Commit Graph

582 Commits

Author SHA1 Message Date
Anders Kaseorg d7981dad62 actions: Split out zerver.actions.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:34 -07:00
Anders Kaseorg bbce879c81 actions: Split out zerver.actions.custom_profile_fields.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:33 -07:00
Anders Kaseorg a29f1b39da actions: Move part into zerver.lib.streams.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:32 -07:00
Anders Kaseorg 385616f27f actions: Split out zerver.actions.realm_emoji.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:31 -07:00
Tim Abbott 2328a81f55 devtools: Remove development-only user creation management commands.
We now have nicer version available not only to developers, and it's
definitely better to deduplicate these.
2022-03-21 12:05:59 -07:00
Tim Abbott 6b00c748fd onboarding: Deduplicate realm creation initial user logic.
We now call this function inside do_create_user(...,
realm_creation=True), which generally improves readability and
robustness of the codebase.

This fixes a bug where this onboarding content was not correctly done
when creating a realm via LDAP, and also will be important as we add
new code paths that might let you create a realm.
2022-03-21 12:05:59 -07:00
Sahil Batra 9345c344d7 user_groups: Add members to the System groups created.
This commit adds users to the appropriate system user group
based on their role. We also change the user groups when
changing role of the user.

We also add migration to add existing users to the appropriate
user groups.

This commit adds update_users_in_full_members_system_group which
is currently used to update the full members group on changing
role of a user. This function will be modified in next commit such
that it can be used to update full members group on changing
waiting_period_threshold setting of realm.
2022-03-14 18:53:46 -07:00
Steve Howell 8f99894302 streams: Extract stream_color library.
This is a pure code move.
2022-03-14 18:01:36 -07:00
Mateusz Mandera f90beae616 zilencer: Drop the index from RemotePushDeviceToken.user_id.
The index isn't used, because our unique_index entries provide better
indexes for the queries.
2022-03-14 17:47:30 -07:00
Mateusz Mandera 0677c90170 zilencer: Change push bouncer API to accept uuids as user identifier.
This is the first step to making the full switch to self-hosted servers
use user uuids, per issue #18017. The old id format is still supported
of course, for backward compatibility.

This commit is separate in order to allow deploying *just* the bouncer
API change to production first.
2022-03-14 17:47:30 -07:00
Alex Vandiver f531f3a27f push_notifications: Drop FCM retries to 2, not 10.
This reverts bc15085098 (which provided
not justification for its change) and moves further, down to 2 retries
from the default of 5.

10 retries, with exponential backoff, is equivalent to sleeping 2^11
seconds, or just about 34 minutes (though the code uses a jitter which
may make this up to 51 minutes).  This is an unreasonable amount of
time to spend in this codepath -- as only one worker is used, and it
is single-threaded, this could effectively block all missed message
notifications for half an hour or longer.

This is also necessary because messages sent through the push bouncer
are sent synchronously; the sending server uses a 30-second timeout,
set in PushBouncerSession.  Having retries which linger longer than
this can cause duplicate messages; the sending server will time out
and re-queue the message in RabbitMQ, while the push bouncer's request
will continue, and may succeed.

Limit to 2 retries (APNS currently uses 3), and results expected max
of 4 seconds of sleep, potentially up to 6.  If this fails, there
exists another retry loop above it, at the RabbitMQ layer (either
locally, or via the remote server's queue), which will result in up to
3 additional retries -- all told, the request will me made to FCM up
to 12 times.
2022-03-08 12:52:58 -08:00
Anders Kaseorg 21cd1c10b3 docs: Add missing space in “time zone”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-24 14:05:12 -08:00
evykassirer 8c3ff92964 populate_db: Add timezones for some test users.
After failing to notice a place where we wanted to hide timezone
information, we decided to add timezones to some of the test
users, so that we can better consider the effects of timezones
when manually testing.

Testing:

* ran populate_db and confirmed users had timezones in the UI
* updated test_populate_db.py
2022-02-22 11:14:58 -08:00
Sahil Batra 947b2b55fe populate_db: Set owners for bots in development and test database.
Since we do not allow to remove owners from bots, it is better
to keep owners for the bots in development environment as well.

We need to change puppeteer tests here because now desdemona
already has bots in dev server and thus "Active bots" section
is opened by default in the settings instead of "Add a new bot"
section.
2022-02-22 10:35:18 -08:00
Anders Kaseorg 1629d6bfb3 python: Reformat with Black 22 (stable).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-18 18:03:13 -08:00
Anders Kaseorg b0ce4f1bce docs: Fix many spelling mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-07 18:51:06 -08:00
Lauryn Menard 3be622ffa7 backend: Add request as parameter to json_success.
Adds request as a parameter to json_success as a refactor towards
making `ignored_parameters_unsupported` functionality available
for all API endpoints.

Also, removes any data parameters that are an empty dict or
a dict with the generic success response values.
2022-02-04 15:16:56 -08:00
Eeshan Garg 3bc0f8c6f9 zilencer: Add endpoint for deactivating remote server registration. 2022-01-21 14:57:04 -08:00
Eeshan Garg 7a1ed9a4cb zilencer: Remove management command for adding remote server.
As @timabbott mentioned on #20577, this command was mostly useful
during early development of the feature, and is no longer needed now
that we have an API for accomplishing the same thing.
2022-01-19 13:13:52 -08:00
Anders Kaseorg 1dfddffc8d profile_request: Use modern Django middleware API.
Fixes “RemovedInDjango40Warning: Passing None for the middleware
get_response argument is deprecated.” from LogRequests().

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 21:03:22 -08:00
Mateusz Mandera 868ed17661 remote_server: Handle invalid server uuid being given authing to API.
get_remote_server_by_uuid (called in validate_api_key) raises
ValidationError when given an invalid UUID due to how Django handles
UUIDField. We don't want that exception and prefer the ordinary
DoesNotExist exception to be raised.
2022-01-04 14:40:49 -08:00
Alex Vandiver 1b395b6403 zilencer: Truncate APNS notifications correctly.
APNs payloads nest the zulip-custom data further than the top level,
as Android notifications do.  This led to APNs data silently never
being truncated; this case was not caught in tests because the mocks
provided the wrong data for the APNs structure.

Adjust to look in the appropriate place within the APNs data, and
truncate that.
2022-01-03 15:24:16 -08:00
Eeshan Garg 0b5324f345 corporate: Add helper for deactivating remote server registrations. 2022-01-03 14:02:48 -08:00
Eeshan Garg 4cc35c339b migrations: Backfill audit log entries for remote server creation.
This is a follow-up to #20408.
2022-01-03 12:58:00 -08:00
Mateusz Mandera 4153b5c517 remote_server: Improve uuid validation at the server/register endpoint.
As explained in the comments in the code, just doing UUID(string) and
catching ValueError is not enough, because the uuid library sometimes
tries to modify the string to convert it into a valid UUID:

>>> a = '18cedb98-5222-5f34-50a9-fc418e1ba972'
>>> uuid.UUID(a, version=4)
UUID('18cedb98-5222-4f34-90a9-fc418e1ba972')
2021-12-31 11:18:01 -08:00
Mateusz Mandera c5c3ab66d6 remote_server: Migrate RemoteZulipServer.uuid to be UUIDField.
Given that these values are uuids, it's better to use UUIDField which is
meant for exactly that, rather than an arbitrary CharField.

This requires modifying some tests to use valid uuids.
2021-12-28 10:11:34 -08:00
Mateusz Mandera e48120fd12 remote_server: Validate zulip_org_id submitted by registering server.
zulip_org_id is supposed to be a UUID, so we want to actually validate
the format, not only check the length.
2021-12-28 10:11:34 -08:00
Tim Abbott 4cb189fc63 settings: Rename TOS_VERSION to TERMS_OF_SERVICE_VERSION.
The previous version was appropriate in a setting where it was only
used for Zulip Cloud, but it's definitely clearer to spell it out.
2021-12-09 17:51:16 -08:00
Eeshan Garg 79e9ba13e2 billing: Add do_change_remote_server_plan_type.
This is a part of the plumbing we need to support billing for
self-hosted customers.

With documentation changes from tabbott.
2021-12-07 10:25:37 -08:00
Aman Agrawal 00d3b88257 realm: Enable spectator access by default in development.
Enable spectator access for test `zulip` realm in developement
setup.

Add option in `do_create_realm` to configure
`enable_spectator_access` field of `Realm`.
2021-11-18 12:55:12 -08:00
Alex Vandiver 800e38016a queue_rate: Output to CSV, and run multiple prefetch values. 2021-11-16 11:48:50 -08:00
Alex Vandiver 6c14978cd1 zilencer: Truncate "remove" notifications from remote servers.
This is 4d055a6695, but for notifications which are received from
remote hosts.
2021-11-10 13:39:35 -08:00
Alex Vandiver 111ee64e36 push_notifications: Pass down the remote server and user-id for logs.
This makes logging more consistent between FCM and APNs codepaths, and
makes clear which user-ids are for local users, and which are opaque
integers namespaced from some remote zulip server.
2021-10-19 22:04:24 -07:00
Alex Vandiver 5bcd3c01cb push_notifications: Add log line with user-id, UUID, and devices.
Being able to determine how many distinct users are getting push
notifications per remote host is useful, as is the distribution of
device counts.  This parallels the log line in
handle_push_notification for push notifications from local realms,
handled via the event queue.
2021-10-19 22:04:24 -07:00
Sahil Batra 828f0e1847 populate_db: Use do_change_user_role to set polonius as guest.
This commit changes populate_db code to set role of polonius
as guest using do_change_user_role. This will make it consistent
with other users also.
2021-10-19 17:02:10 -07:00
Eeshan Garg b325a4f1be realm: Rename plan type constants to be more descriptive.
It is confusing to have the plan type constants not be namespaced
by the thing they represent. We already have a namespacing
convention in place for constants, so we should use it for
Realm.plan_type as well.
2021-10-19 12:20:39 -07:00
Vishnu KS fcab2ea5f7 billing: Add command for switching plans from Standard to Plus. 2021-10-15 17:27:50 -07:00
Mateusz Mandera 0af7c84c99 push_notifs: Log the number of devices notification was sent to. 2021-09-29 15:50:06 -07:00
Anders Kaseorg 729f70142f populate_db: Fix naive datetime RuntimeWarning.
Fixes these warnings from populate_db:

/srv/zulip-py3-venv/lib/python3.6/site-packages/django/db/models/fields/__init__.py:1419: RuntimeWarning: DateTimeField Draft.last_edit_time received a naive datetime (2021-09-10 23:33:15.063608) while time zone support is active.
  RuntimeWarning)
/srv/zulip-py3-venv/lib/python3.6/site-packages/django/db/models/fields/__init__.py:1419: RuntimeWarning: DateTimeField Draft.last_edit_time received a naive datetime (2021-09-10 23:33:15.065517) while time zone support is active.
  RuntimeWarning)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-10 17:02:50 -07:00
Anders Kaseorg a4dbc1edd4 docs: Format Markdown with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
PIG208 53888e5a26 request: Refactor ZulipRequestNotes to RequestNotes.
This utilizes the generic `BaseNotes` we added for multipurpose
patching. With this migration as an example, we can further support
more types of notes to replace the monkey-patching approach we have used
throughout the codebase for type safety.
2021-09-03 08:48:45 -07:00
PIG208 bb36bd8b6b profile_request: Replace MockRequest with HostRequestMock.
Since previously we use a custom mock request that doesn't inherit
HttpRequest, there are some typing problems when we use it.
2021-08-20 05:54:19 -07:00
PIG208 460119986b management: Fix typing for management scripts.
There are some remaining errors related to the django `Manager[T]` and
the `List[T]` type that we use to annotate the `Manage[T]` objects.
2021-08-20 05:54:18 -07:00
Anders Kaseorg 4206e5f00b python: Remove locally dead code.
These changes are all independent of each other; I just didn’t feel
like making dozens of commits for them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 5483ebae37 python: Convert "".format to Python 3.6 f-strings.
Generated automatically by pyupgrade.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-02 15:53:52 -07:00
Tim Abbott 472c55a1ff populate_db: Create some drafts in the initial database. 2021-07-30 10:41:52 -07:00
PIG208 e517f967b8 typing: Amend inaccurate type annotations.
This amend some type annotations that turn out to be inaccurate with
django-stubs.
2021-07-27 12:17:00 -07:00
PIG208 434b7e360a populate_db: Remove unused `output`.
The helper function `generate_and_send_messages` receives
`self.stdout.write` from `jobs`, but it is never actually used.
2021-07-26 15:09:07 -07:00
PIG208 d2af20eb2a typing: Do explicit type conversion when appropriate.
When calling some functions or assigning values to certain attributes,
the arguments/right operand do not match the exact type that the
functions/attributes expect, and thus we fix that by converting types
beforehand.
2021-07-26 15:09:07 -07:00
Dinesh e4fb2883b4 populate_db: Re-enable push_notifications logger.
We were disabling push_notifications logger but weren't enabling
it back. This caused failures on porting logging mocks to assertLogs
as assertLogs expects a log to be generated.

See 9c224ccdd3 for why we disable
these. (To avoid logs spam from push_notifications_logger)
2021-07-26 14:46:01 -07:00