Commit Graph

10830 Commits

Author SHA1 Message Date
Hashir Sarwar dcbd3e486f stream_subscription: Remove unused TypedDict `SubInfo`. 2020-02-10 14:04:22 -08:00
Steve Howell 2ff41bf9e5 /json/users: Use field.realm for realm lookup.
This avoids an unnecessary join to UserProfile.

To verify this, you can do `print(queries)` in the
`test_get_custom_profile_fields_from_api` test.  It's
kinda noisy, so I excerpted them below...

Before:

    SELECT ...
    FROM "zerver_customprofilefieldvalue"
    INNER JOIN "zerver_userprofile" ON ("zerver_customprofilefieldvalue"."user_profile_id" = "zerver_userprofile"."id")
    INNER JOIN "zerver_customprofilefield" ON ("zerver_customprofilefieldvalue"."field_id" = "zerver_customprofilefield"."id")
    WHERE "zerver_userprofile"."realm_id" = 2

After:

    SELECT ...
    FROM "zerver_customprofilefieldvalue"
    INNER JOIN "zerver_customprofilefield" ON ("zerver_customprofilefieldvalue"."field_id" = "zerver_customprofilefield"."id")
    WHERE "zerver_customprofilefield"."realm_id" = 2'

I don't have any way to measure the two queries with
realistic data, but I would assume the second
query is significantly faster on most of our instances,
since CustomProfileField should be tiny.
2020-02-09 22:04:02 -08:00
Steve Howell 9303c386b8 tests: Count queries for /json/users.
I am trying to optimize a query in this endpoint.
I don't think I'll actually reduce the number of
queries, but I wanted to capture the query and
this was the easiest way to do it, so might as
well check in the code! :)
2020-02-09 22:04:02 -08:00
Steve Howell 01f180d042 minor: Remove unused line of code in get_raw_user_data().
The line removed here is a noop, as both sides of the
immediately following conditional reassign the
same variable.

This harmless cruft was the result of the recent commit
1ae5964ab8, which added
support for single-user GETs.
2020-02-09 22:04:02 -08:00
Tim Abbott 986706c7e5 tornado: Use common code for copying headers.
This fixes a bug where our asynchronous requests were only copying the
Content-Type header (i.e. the one case where we're noticed) from the
Django HttpResponse.  I'm not sure what the impact of this would be;
the rate-limiting headers rarely come up when breaking a long-polled
request.  But it seems clearly an improvement to do this in a
consistent fashion.

Only the headers piece is a change; in Tornado

    self.finish(x)

is equivalent to:

    self.write(x)
    self.finish()
2020-02-07 16:14:19 -08:00
Tim Abbott 224a73a3ec tornado: Extract a function for writing Tornado responses.
This increases the readability of what's happening in our core Tornado
handlers code, as well as making this logic reusable.
2020-02-07 16:13:49 -08:00
Tim Abbott 5305e8af85 tornado: Extract convert_tornado_request_to_django_request. 2020-02-07 16:03:58 -08:00
Tim Abbott fc58ae117a handlers: Rename confusingly named response to result_dict.
This should somewhat increase the readability of zulip_finish.
2020-02-07 16:03:58 -08:00
Vishnu KS 4572be8c27 api: Rename subject_links to topic_links.
Fixes #13588
2020-02-07 14:35:22 -08:00
Tim Abbott 84edb5c516 test_fixtures: Fix buggy reuse of status_dir between databases.
Apparently, the arguments passed to template_database_status were
incorrect for the manual testing development database, in that we
didn't pass a status_dir when calling into that code from provision.

The result was that provisioning before running `test-backend` would
ignore changes to the list of check_files (etc.) made after rebasing,
and vice versa.

The cleanest fix is to compute status_dir from other values passed in;
I'm also going to open a follow-up issue for creating a better overall
interface here.
2020-02-07 13:33:08 -08:00
akashaviator 1ae5964ab8 api: Add an api endpoint for GET /users/{id}
This adds a new API endpoint for querying basic data on a single other
user in the organization, reusing the existing infrastructure (and
view function!) for getting data on all users in an organization.

Fixes #12277.
2020-02-07 10:36:31 -08:00
Tim Abbott e39840c705 users: Add read-only mode for access_user_by_id.
We've be using this in the upcoming GET /users/{id} method.
2020-02-07 10:36:31 -08:00
Tim Abbott aa9286a1f9 users: Move query into caller of get_custom_profile_field_values.
This will be useful for supporting a smaller query for a single user.
2020-02-07 10:36:31 -08:00
Tim Abbott 79e5dd1374 users: Rename get_raw_user_data user parameter to acting_user.
This is for improved clarity as we extend this function to take
multiple user objects.
2020-02-07 10:36:31 -08:00
Steve Howell 7e99e7feb2 presence: Extract get_legacy_user_info.
This code is a bit flatter and just preps the data
for a single user.  There is never any interaction
between the data for user A and user B, so we can
mostly avoid complicated nested data structures
and do most of the data-crunching on a per-user basis.

We also do an explicit sort of the data before
running it through groupby.  The explicit sort
simplifies how we calculate `most_recent_info`
and also avoids needing to add `dt` to an intermediate
data structure.

Finally, when it comes to the individual client data,
the code has relied on the assumption that there is
only one row per client, which I believe to be true,
but now the code is more explicit about that.
2020-02-06 17:16:22 -08:00
Steve Howell bf3baa14ac presence: Rename get_status_dict_by_user(). 2020-02-06 17:16:22 -08:00
Steve Howell 675f8514e8 presence: Rename get_status_dict().
We renamed this to get_presences_for_realm(),
and we have the caller pass in realm, not
user_profile.
2020-02-06 17:16:22 -08:00
Steve Howell 363e6bf239 presence: Move get_status_dicts_for_rows(). 2020-02-06 17:16:22 -08:00
Steve Howell 36fba1076f presence: Move get_status_dict_by_user. 2020-02-06 17:16:22 -08:00
Steve Howell 6f027d84a9 presence: Move get_status_dict_by_realm. 2020-02-06 17:16:22 -08:00
Steve Howell 703338dfa3 presence: Extract lib/presence.py.
This will make more sense when we pull some
code out of the model.
2020-02-06 17:16:22 -08:00
Steve Howell a5093be867 presence: Rename get_status_list.
The word "status" is vague, and this isn't
actually returning a list, so we now name it
get_presence_response.

I originally was gonna rename this to
get_presence_dict, but there's a function
called get_status_dict that returns a subset
of the response, so I think it's a bit more
clear that this is the bigger dict that
actually gets sent back.
2020-02-06 17:16:22 -08:00
Steve Howell 8a1fb2dcd6 presence: Calculate server_timestamp slightly earlier.
We want to err on the side of server_timestamp being
old, since we may eventually use this to make responses
just include incremental changes, and we don't want a
time window (however small) when we miss presence rows.
The clients will be able to deal with duplicate data
to the extent that the time windows are overlapping.

Also, extracting the other local var here
(for `presences`) will set up a subsequent commit
where we re-format the data for clients with
slim_presence=True.
2020-02-06 17:16:22 -08:00
Tim Abbott b5cc1a3afe home: Finish removing assumption that user_profile isn't None. 2020-02-05 16:16:33 -08:00
Tim Abbott be02604fe5 home: Refactor logic for several more settings.
The main functional change is there's now a code path for
`user_profile is None`.
2020-02-05 16:13:29 -08:00
Tim Abbott 5f8af84ad1 home: Refactor emojiset and billing logic for None UserProfile. 2020-02-05 16:09:03 -08:00
Tim Abbott 7c0a98754a home: Refactor logic for show_invites and show_add_streams. 2020-02-05 16:05:02 -08:00
Tim Abbott fc4400cfa7 home: Remove unused avatar_url context parameter. 2020-02-05 16:05:02 -08:00
Tim Abbott 7044aa8e5d home: Extract updates to last_reminder to a function. 2020-02-05 16:05:02 -08:00
Tim Abbott 85690ce649 home: Extract handling of narrow windows to a function. 2020-02-05 16:05:02 -08:00
Tim Abbott 42d529e749 home: Extract terms of service check to a function. 2020-02-05 16:05:01 -08:00
Tim Abbott 7032f49f8e exceptions: Move default json_unauthorized string to response.py.
This small refactor should make it easier to reuse this exception for
other situations as well.
2020-02-05 15:40:10 -08:00
Tim Abbott 2aab71e153 event_queue: Fix confusing event_queue.push interface.
In e3ad9baf1d, we introduced yet another
bug where we incorrectly shared event dictionaries between multiple
queues.

Fortunately, the logging that reports on "event was not in the queue"
issues worked and detected this on chat.zulip.org, but this is a clear
indication that the comments we have around this system were not
sufficient to produce correct behavior.

We fix this by changing event_queue.push, the code that mutates the
event dictionaries, to do the shallow copies itself.  The only
downside here is process_message_event, a relatively low-traffic code
path, does an extra per-queue dictionary copy.  Given that presence,
heartbeat, and message reading events are likely more traffic and
dealing with HTTP is likely much more expensive than a dictionary
copy, this probably doesn't matter performance-wise.

(And if profiling later finds it is, there are potential workarounds
like passing a skip_copy argument we can do).
2020-02-05 12:40:01 -08:00
Anders Kaseorg 8e5a45267d test_classes: Use a valid (but reserved as fictional) phone number.
django-phonenumber-field 2.4.0 adds tighter phone number validation
that rejects +12223334444 for having an invalid area code.  This was
reverted in 4.0.0, but django-two-factor-auth still requires <3.99.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-05 12:38:10 -08:00
Anders Kaseorg 4d49a20430 requirements: Upgrade django-sendfile2 from 0.4.3 to 0.5.1.
The module was renamed from sendfile to django_sendfile.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-05 12:38:10 -08:00
Ryan Rehman 174b2abcfd settings: Migrate to stream_post_policy structure.
This commit includes a new `stream_post_policy` setting,
by replacing the `is_announcement_only` field from the Stream model,
which is done by mirroring the structure of the existing
`create_stream_policy`.

It includes the necessary schema and database migrations to migrate
the is_announcement_only boolean field to stream_post_policy,
a smallPositiveInteger field similar to many other settings.

This change is done to allow organization administrators to restrict
new members from creating and posting to a stream. However, this does
not affect admins who are new members.

With many tweaks by tabbott to documentation under /help, etc.

Fixes #13616.
2020-02-04 17:08:08 -08:00
Mateusz Mandera 30d02c2e2c test_fixtures: app_label should be a positional arg in call_command.
We were incorrectly passing it as a kwarg, which would cause an
exception on Django 2.
2020-02-04 12:46:53 -08:00
Mateusz Mandera 5de832283b test_auth_backends: Remove logger mocking that will fail on Django 2.2.
On Django 2.2 there is no longer a logger object in that module, so it's
best to remove this mocking as it's not essential to the tests.
2020-02-04 12:46:53 -08:00
Mateusz Mandera 0e7c97378e is_safe_url: Use allowed_hosts instead of depreciated host argument.
Judging by comparing django 1.11 with django 2.2 code of this function,
this shouldn't change any behavior.
2020-02-04 12:46:53 -08:00
Steve Howell e3ad9baf1d presence: Add process_presence_event.
This lets us conditionally remove the email
field from a presence event if the client
has registered with the slim_presence flag.
2020-02-04 12:30:36 -08:00
Steve Howell 9847d4d9a3 refactor: Use user_id in get_status_dict_by_user.
This avoids a needless user lookup in apply_event.
2020-02-04 12:30:36 -08:00
Steve Howell a672a00677 presence: Add user_id to presence event.
In a later commit, we will eliminate email for
clients who have set slim_presence as their
preference.
2020-02-04 12:30:36 -08:00
Steve Howell bf9144ff69 presence: Add slim_presence flag.
This flag affects page_params and the
payload you get back from POSTs to this
url:

    users/me/presence

The flag does not yet affect the
presence events that get sent to a
client.
2020-02-04 12:30:34 -08:00
Vishnu KS df5345705c import: Support importing team icon from slack. 2020-02-03 14:09:05 -08:00
Vishnu Ks 5dfd4ea38d export: Remove unused parameter from _get_exported_s3_record. 2020-02-03 14:09:05 -08:00
Vishnu Ks 5a59bf329e import: Skip setting user_profile_id metadata only if unavailable. 2020-02-03 14:09:05 -08:00
Vishnu Ks 2ea53a347a import: Support importing realm icon and logo.
Fixes #11216
2020-02-03 14:09:05 -08:00
Vishnu Ks af3a37b58b upload: Refactor out realm_avatar_and_logo_path function. 2020-02-03 14:09:05 -08:00
Tim Abbott df110e8ff9 test_fixtures: Note populate_db depends on server_initialization.py.
This should ensure that folks rebasing past this commit from an older
database model get their database rebuilt in the way that will
match the test_subs.py query count of 40.
2020-02-03 10:38:04 -08:00
Ryan Rehman 3dc7d60ffe muting: Record DateTime when a Topic is muted.
This includes the necessary migration to add
the date_muted field to the MutedTopic class
and populates it with a hard coded value.
2020-02-02 20:49:53 -08:00