Small follow-up to d86e4ac34d.
get_ makes it sound like it doesn't have side-effects, when these are
actually much like the django ORM .get_or_create function.
These are used for creating huddles and private messages (and some
UserPresence objects). It'd be really weird, and potentially create some
Messages that break our assumptions, for this to end up involving users
in multiple realms.
I believe currently this hasn't been happening, because when
this line runs, there are only users in "zulip" realm and system bots in
"zulipinternal" - but the query has been excluding bots already.
Still, this query should be explicit about grabbing users from a single
realm. This will also be helpful for the work adding the denormalized
Message.realm field - so that the realm of Message objects that get
manually created in generate_and_send_messages can be simply set to
"zulip" with confidence that it's correct.
In the presence of **kwargs, this is required by the Concatenate type
expected by default_never_cache_responses.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
In 5c49e4ba06, we neglected to include
the CSRF and caching decorators required for all API views in the new
remote_server_dispatch function.
I'm not sure why our automated tests didn't catch this, but this made
the remote server API endpoints nonfunctional in a production
environment.
This change incorporate should_rate_limit into rate_limit_user and
rate_limit_request_by_ip. Note a slight behavior change to other callers
to rate_limit_request_by_ip is made as we now check if the client is
eligible to be exempted from rate limiting now, which was previously
only done as a part of zerver.lib.rate_limiter.rate_limit.
Now we mock zerver.lib.rate_limiter.RateLimitedUser instead of
zerver.decorator.rate_limit_user in
zerver.tests.test_decorators.RateLimitTestCase, because rate_limit_user
will always be called but rate limit only happens the should_rate_limit
check passes;
we can continue to mock zerver.lib.rate_limiter.rate_limit_ip, because the
decorated view functions call rate_limit_request_by_ip that calls
rate_limit_ip when the should_rate_limit check passes.
We need to mock zerver.decorator.rate_limit_user for SkipRateLimitingTest
now because rate_limit has been removed. We don't need to mock
RateLimitedUser in this case because we are only verifying that
the skip_rate_limiting flag works.
To ensure coverage in add_logging_data, a new test case is added to use
a web_public_view (which decorates the view function with
add_logging_data) with a new flag to check_rate_limit_public_or_user_views.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This allows us to avoid importing from zilencer conditionally in
zerver.lib.rate_limiter, as we make rate limiting self-contained now.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This refactors the test case alongside, since normal views accessed by
remote server do not get rate limited by remote server anymore.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`remote_server_path` allows us to get rid of all the `validate_entity`
calls in `zilencer.views` and remove all the `Union` type annotations
in the signatures of the authenticated view functions.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This allows us to separate the zilencer paths from other JSON paths,
with explicit type annotation expecting `RemoteZulipServer` as the
second parameter of the handler using
authenticated_remote_server_view.
The test case is also updated to remove a test for a situation that no
longer occurs anymore, since we don't perform subdomain checks on
remote servers.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Adds an API endpoint for accessing read receipts for other users, as
well as a modal UI for displaying that information.
Enables the previously merged privacy settings UI for managing whether
a user makes read receipts data available to other users.
Documentation is pending, and we'll likely want to link to the
documentation with help_settings_link once it is complete.
Fixes#3618.
Co-authored-by: Tim Abbott <tabbott@zulip.com>
This commit modifies bulk_create_users to add the users to the
respective system groups. And due to this change, now bots in
development environment are also added to system groups.
Tests are changed accordingly as more UserGroupMembeship objects
are created.
`_cache` is not an attribute defined on `BaseCache`, but an
implementation detail of django_bmemcache.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
The field_data sent from client while creating a select
type field is a dict with a number as key.
In development database the field data for "Favorite editor"
field was of different form where the option label was used
as key in the dict.
This commit fixes it to be of the same as it is when creating
a field from web-app. As a result, we also need to update
the tests and this commit also update field_data for other
select-type fields.
We increase the total number of messages, since increasing the number
of topics would otherwise have the side effect of making it hard to
find longer conversations.
We do not allow keeping vacant private streams as we deactivate them
when all users are unsubscribed, so it is better to add at least one
user in the initial private stream created while creating realm.
This commit adds code to copy the realm-level default of
settings while creating users through bulk_create_users.
We do not directly call 'copy_default_settings' as it
calls ".save()" but here we want to bulk_create the objects
for efficiency.
We also add the code to set realm-default of enter_sends as
True for the Zulip dev server as done in 754b547e8 and thus
we remove enter_sends argument from create_user_profile as
it is of no use now.
Bots don't generally do API requests to mark messages as read. If they
did, it's likely because the developer of the bot wants them to appear
in read receipts or similar (E.g. as an indication of what messages
have been processed).
So we should avoid setting the read flag on bot messages the test
database.
76deb30312 changed this to not just be the URL, but rather a
prefixed hash of the URL, but failed to update this location which
wrote to it. This meant that this pre-population step was writing to
the wrong keys in the durable cache, and thus ineffective.
Then, da33b72848 switched the cache to be in-memory, making this
write to the wrong keys in an in-process memory store. There is no
way to pre-fill this sort of cache, except at server start-up.
Finally, and most fundamentally, 8c0c9ca7a4 then disabled
`inline_url_embed_preview` by default, making the code entirely moot.
Remove the triply-unnecessary code.
We now call this function inside do_create_user(...,
realm_creation=True), which generally improves readability and
robustness of the codebase.
This fixes a bug where this onboarding content was not correctly done
when creating a realm via LDAP, and also will be important as we add
new code paths that might let you create a realm.
This commit adds users to the appropriate system user group
based on their role. We also change the user groups when
changing role of the user.
We also add migration to add existing users to the appropriate
user groups.
This commit adds update_users_in_full_members_system_group which
is currently used to update the full members group on changing
role of a user. This function will be modified in next commit such
that it can be used to update full members group on changing
waiting_period_threshold setting of realm.
This is the first step to making the full switch to self-hosted servers
use user uuids, per issue #18017. The old id format is still supported
of course, for backward compatibility.
This commit is separate in order to allow deploying *just* the bouncer
API change to production first.
This reverts bc15085098 (which provided
not justification for its change) and moves further, down to 2 retries
from the default of 5.
10 retries, with exponential backoff, is equivalent to sleeping 2^11
seconds, or just about 34 minutes (though the code uses a jitter which
may make this up to 51 minutes). This is an unreasonable amount of
time to spend in this codepath -- as only one worker is used, and it
is single-threaded, this could effectively block all missed message
notifications for half an hour or longer.
This is also necessary because messages sent through the push bouncer
are sent synchronously; the sending server uses a 30-second timeout,
set in PushBouncerSession. Having retries which linger longer than
this can cause duplicate messages; the sending server will time out
and re-queue the message in RabbitMQ, while the push bouncer's request
will continue, and may succeed.
Limit to 2 retries (APNS currently uses 3), and results expected max
of 4 seconds of sleep, potentially up to 6. If this fails, there
exists another retry loop above it, at the RabbitMQ layer (either
locally, or via the remote server's queue), which will result in up to
3 additional retries -- all told, the request will me made to FCM up
to 12 times.
After failing to notice a place where we wanted to hide timezone
information, we decided to add timezones to some of the test
users, so that we can better consider the effects of timezones
when manually testing.
Testing:
* ran populate_db and confirmed users had timezones in the UI
* updated test_populate_db.py
Since we do not allow to remove owners from bots, it is better
to keep owners for the bots in development environment as well.
We need to change puppeteer tests here because now desdemona
already has bots in dev server and thus "Active bots" section
is opened by default in the settings instead of "Add a new bot"
section.
Mateusz Mandera
Anders Kaseorg
Tim Abbott
Zixuan James Li
Dinesh
Sahil Batra
Zixuan James Li
evykassirer
Alex Vandiver
Steve Howell