Rather than pass around a list of message objects in-memory, we
instead keep the same constructed QuerySet which includes the later
propagated messages (if any), and use that same query to pick out
affected Attachment objects, rather than limiting to the set of ids.
This is not necessarily a win -- the list of message-ids *may* be very
long, and thus the query may be more concise, easier to send to
PostgreSQL, and faster for PostgreSQL to parse. However, the list of
ids is almost certainly better-indexed.
After processing the move, the QuerySet must be re-defined as a search
of ids (and possibly a very long list of such), since there is no
other way which is guaranteed to correctly single out the moved
messages. At this point, it is mostly equivalent to the list of
Message objects, and certainly takes no less memory.
This applies access restrictions in SQL, so that individual messages
do not need to be walked one-by-one. It only functions for stream
messages.
Use of this method significantly speeds up checks if we moved "all
visible messages" in a topic, since we no longer need to walk every
remaining message in the old topic to determine that at least one was
visible to the user. Similarly, it significantly speeds up merging
into existing topics, since it no longer must walk every message in
the new topic to determine if the user could see at least one.
Finally, it unlocks the ability to bulk-update only messages the user
has access to, in a single query (see subsequent commit).
This is a preparatory commit that refactors the check_update_message
method to extract the checks containing whether a user can edit the
message or not into a separate method -validate_message_content_edit,
so that it can be re used later.
This logic was apparently missed when we implemented private streams
with shared history; the correct check is to look at whether the user
can access message history in the stream, which used to be equivalent
to whether it's a private stream.
This commit removes the stale 'email_gateway' parameter
from 'do_send_messages' function.
This should have been removed in 6c473ed75f,
when the call to 'build_message_send_dict' was removed
from 'do_send_messages'.
The endpoint was lacking validation that the authentication_methods dict
submitted by the user made sense. So e.g. it allowed submitting a
nonsense key like NoSuchBackend or modifying the realm's configured
authentication methods for a backend that's not enabled on the server,
which should not be allowed.
Both were ultimately harmless, because:
1. Submitting NoSuchBackend would luckily just trigger a KeyError inside
the transaction.atomic() block in do_set_realm_authentication_methods
so it would actually roll back the database changes it was trying to
make. So this couldn't actually create some weird
RealmAuthenticationMethod entries.
2. Silently enabling or disabling e.g. GitHub for a realm when GitHub
isn't enabled on the server doesn't really change anything. And this
action is only available to the realm's admins to begin with, so
there's no attack vector here.
test_supported_backends_only_updated wasn't actually testing anything,
because the state it was asserting:
```
self.assertFalse(github_auth_enabled(realm))
self.assertTrue(dev_auth_enabled(realm))
self.assertFalse(password_auth_enabled(realm))
```
matched the desired state submitted to the API...
```
result = self.client_patch(
"/json/realm",
{
"authentication_methods": orjson.dumps(
{"Email": False, "Dev": True, "GitHub": False}
).decode()
},
)
```
so we just replace it with a new test that tests the param validation.
This leads to significant speedups. In a test, with 100 random unique
event classes, the old code processed a batch of 100 rows (on average
66-ish unique in the batch) in 0.45 seconds. Doing this in a single
query processes the same batch in 0.0076 seconds.
This is preparatory work towards adding a Topic model.
We plan to use the local variable name as 'topic' for
the Topic model objects.
Currently, we use *topic as the local variable name for
topic names.
We rename local variables of the form *topic to *topic_name
so that we don't need to think about type collisions in
individual code paths where we might want to talk about both
Topic objects and strings for the topic name.
Rename and restructure these comparison variables such that we don't
have a possibly impossible case for presence.last_connected_time being
None.
Fixes#25498.
We previously used get_accessible_user_ids to check whether the
sender can access all DM recipients, which was not efficient as
it queries the Message table. This commit updates the code to
make sure we use get_inaccessible_user_ids which is much more
efficient as it limits the queries to only DM recipients and
also queries the Message table only if needed.
This can still be optimized further as mentioned in #27835 but
this commit is a nice first step.
This fixes a bug introduced in
6f93ab72c0 where deactivating a realm
would fail with an exception that sessions cannot be cleared inside
database transactions.
Actions that change the number of user counts adds a deferred_work
queue processor job immediately update the billing service about your
change.
This helps to avoid having users see stale state for how many
users they have when trying to pay.
This is a rename of the previous
enqueue_register_realm_with_push_bouncer_if_needed but is clearer
about the fact that this will also upload audit logs if available.
This is the only operating editing audit logs not already using a
transaction, and having it do so will simplify an upcoming interface
to be able to assume it is always inside a transaction.
This commit adds code to not include original details of senders like
name, email and avatar url in the message objects sent through events
and in the response of endpoint used to fetch messages.
This is the last major commit for the project to add support for
limiting guest access to an entire organization.
Fixes#10970.
Earlier, the event sent when an onboarding step (hotspot till now)
is marked as read generated an event with type='hotspots' and
'hotspots' named array in it.
This commit renames the type to 'onboarding_steps' and the array
to 'onboarding_steps' to reflect the fact that it'll also contain
data for elements other than hotspots.
This commit adds a new endpoint 'users/me/onboarding_steps'
deprecating the older 'users/me/hotspots' to mark hotspot as read.
We also renamed the view `mark_hotspot_as_read` to
`mark_onboarding_step_as_read`.
Reason: Our plan is to make this endpoint flexible to support
other types of UI elements not just restricted to hotspots.
Alex Vandiver
Tim Abbott
roanster007
shashank-23002
Prakhar Pratyush
Mateusz Mandera
Anders Kaseorg
Sahil Batra
Vector73