Commit Graph

4792 Commits

Author SHA1 Message Date
Steve Howell b33552997e cross realm bots: Simplify notify_new_user.
Instead of having `notify_new_user` delegate
all the heavy lifting to `send_signup_message`,
we just rename `send_signup_message` to be
`notify_new_user` and remove the one-line
wrapper.

We remove a lot of obsolete complexity:

    - `internal` was no longer ever set to True
      by real code, so we kill it off as well
      as well as killing off the internal_blurb code
      and the now-obsolete test

    - the `sender` parameter was actually an
      email, not a UserProfile, but I think
      that got past mypy due to the caller
      passing in something from settings.py

    - we were only passing in NOTIFICATION_BOT
      for the sender, so we just hard code
      that now

    - we eliminate the verbose
      `admin_realm_signup_notifications_stream`
      parameter and just hard code it to
      "signups"

    - we weren't using the optional realm
      parameter

There's also a long ugly comment in
`get_recipient_info` related to this code
that I amended for now.
We should try to take action in a subsequent
commit.
2020-02-10 15:45:13 -08:00
Hashir Sarwar dcbd3e486f stream_subscription: Remove unused TypedDict `SubInfo`. 2020-02-10 14:04:22 -08:00
Steve Howell 2ff41bf9e5 /json/users: Use field.realm for realm lookup.
This avoids an unnecessary join to UserProfile.

To verify this, you can do `print(queries)` in the
`test_get_custom_profile_fields_from_api` test.  It's
kinda noisy, so I excerpted them below...

Before:

    SELECT ...
    FROM "zerver_customprofilefieldvalue"
    INNER JOIN "zerver_userprofile" ON ("zerver_customprofilefieldvalue"."user_profile_id" = "zerver_userprofile"."id")
    INNER JOIN "zerver_customprofilefield" ON ("zerver_customprofilefieldvalue"."field_id" = "zerver_customprofilefield"."id")
    WHERE "zerver_userprofile"."realm_id" = 2

After:

    SELECT ...
    FROM "zerver_customprofilefieldvalue"
    INNER JOIN "zerver_customprofilefield" ON ("zerver_customprofilefieldvalue"."field_id" = "zerver_customprofilefield"."id")
    WHERE "zerver_customprofilefield"."realm_id" = 2'

I don't have any way to measure the two queries with
realistic data, but I would assume the second
query is significantly faster on most of our instances,
since CustomProfileField should be tiny.
2020-02-09 22:04:02 -08:00
Steve Howell 01f180d042 minor: Remove unused line of code in get_raw_user_data().
The line removed here is a noop, as both sides of the
immediately following conditional reassign the
same variable.

This harmless cruft was the result of the recent commit
1ae5964ab8, which added
support for single-user GETs.
2020-02-09 22:04:02 -08:00
Vishnu KS 4572be8c27 api: Rename subject_links to topic_links.
Fixes #13588
2020-02-07 14:35:22 -08:00
Tim Abbott 84edb5c516 test_fixtures: Fix buggy reuse of status_dir between databases.
Apparently, the arguments passed to template_database_status were
incorrect for the manual testing development database, in that we
didn't pass a status_dir when calling into that code from provision.

The result was that provisioning before running `test-backend` would
ignore changes to the list of check_files (etc.) made after rebasing,
and vice versa.

The cleanest fix is to compute status_dir from other values passed in;
I'm also going to open a follow-up issue for creating a better overall
interface here.
2020-02-07 13:33:08 -08:00
akashaviator 1ae5964ab8 api: Add an api endpoint for GET /users/{id}
This adds a new API endpoint for querying basic data on a single other
user in the organization, reusing the existing infrastructure (and
view function!) for getting data on all users in an organization.

Fixes #12277.
2020-02-07 10:36:31 -08:00
Tim Abbott e39840c705 users: Add read-only mode for access_user_by_id.
We've be using this in the upcoming GET /users/{id} method.
2020-02-07 10:36:31 -08:00
Tim Abbott aa9286a1f9 users: Move query into caller of get_custom_profile_field_values.
This will be useful for supporting a smaller query for a single user.
2020-02-07 10:36:31 -08:00
Tim Abbott 79e5dd1374 users: Rename get_raw_user_data user parameter to acting_user.
This is for improved clarity as we extend this function to take
multiple user objects.
2020-02-07 10:36:31 -08:00
Steve Howell 7e99e7feb2 presence: Extract get_legacy_user_info.
This code is a bit flatter and just preps the data
for a single user.  There is never any interaction
between the data for user A and user B, so we can
mostly avoid complicated nested data structures
and do most of the data-crunching on a per-user basis.

We also do an explicit sort of the data before
running it through groupby.  The explicit sort
simplifies how we calculate `most_recent_info`
and also avoids needing to add `dt` to an intermediate
data structure.

Finally, when it comes to the individual client data,
the code has relied on the assumption that there is
only one row per client, which I believe to be true,
but now the code is more explicit about that.
2020-02-06 17:16:22 -08:00
Steve Howell bf3baa14ac presence: Rename get_status_dict_by_user(). 2020-02-06 17:16:22 -08:00
Steve Howell 675f8514e8 presence: Rename get_status_dict().
We renamed this to get_presences_for_realm(),
and we have the caller pass in realm, not
user_profile.
2020-02-06 17:16:22 -08:00
Steve Howell 363e6bf239 presence: Move get_status_dicts_for_rows(). 2020-02-06 17:16:22 -08:00
Steve Howell 36fba1076f presence: Move get_status_dict_by_user. 2020-02-06 17:16:22 -08:00
Steve Howell 6f027d84a9 presence: Move get_status_dict_by_realm. 2020-02-06 17:16:22 -08:00
Steve Howell 703338dfa3 presence: Extract lib/presence.py.
This will make more sense when we pull some
code out of the model.
2020-02-06 17:16:22 -08:00
Tim Abbott 7c0a98754a home: Refactor logic for show_invites and show_add_streams. 2020-02-05 16:05:02 -08:00
Tim Abbott 7032f49f8e exceptions: Move default json_unauthorized string to response.py.
This small refactor should make it easier to reuse this exception for
other situations as well.
2020-02-05 15:40:10 -08:00
Anders Kaseorg 8e5a45267d test_classes: Use a valid (but reserved as fictional) phone number.
django-phonenumber-field 2.4.0 adds tighter phone number validation
that rejects +12223334444 for having an invalid area code.  This was
reverted in 4.0.0, but django-two-factor-auth still requires <3.99.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-05 12:38:10 -08:00
Ryan Rehman 174b2abcfd settings: Migrate to stream_post_policy structure.
This commit includes a new `stream_post_policy` setting,
by replacing the `is_announcement_only` field from the Stream model,
which is done by mirroring the structure of the existing
`create_stream_policy`.

It includes the necessary schema and database migrations to migrate
the is_announcement_only boolean field to stream_post_policy,
a smallPositiveInteger field similar to many other settings.

This change is done to allow organization administrators to restrict
new members from creating and posting to a stream. However, this does
not affect admins who are new members.

With many tweaks by tabbott to documentation under /help, etc.

Fixes #13616.
2020-02-04 17:08:08 -08:00
Mateusz Mandera 30d02c2e2c test_fixtures: app_label should be a positional arg in call_command.
We were incorrectly passing it as a kwarg, which would cause an
exception on Django 2.
2020-02-04 12:46:53 -08:00
Mateusz Mandera 0e7c97378e is_safe_url: Use allowed_hosts instead of depreciated host argument.
Judging by comparing django 1.11 with django 2.2 code of this function,
this shouldn't change any behavior.
2020-02-04 12:46:53 -08:00
Steve Howell e3ad9baf1d presence: Add process_presence_event.
This lets us conditionally remove the email
field from a presence event if the client
has registered with the slim_presence flag.
2020-02-04 12:30:36 -08:00
Steve Howell 9847d4d9a3 refactor: Use user_id in get_status_dict_by_user.
This avoids a needless user lookup in apply_event.
2020-02-04 12:30:36 -08:00
Steve Howell a672a00677 presence: Add user_id to presence event.
In a later commit, we will eliminate email for
clients who have set slim_presence as their
preference.
2020-02-04 12:30:36 -08:00
Steve Howell bf9144ff69 presence: Add slim_presence flag.
This flag affects page_params and the
payload you get back from POSTs to this
url:

    users/me/presence

The flag does not yet affect the
presence events that get sent to a
client.
2020-02-04 12:30:34 -08:00
Vishnu Ks 5dfd4ea38d export: Remove unused parameter from _get_exported_s3_record. 2020-02-03 14:09:05 -08:00
Vishnu Ks 5a59bf329e import: Skip setting user_profile_id metadata only if unavailable. 2020-02-03 14:09:05 -08:00
Vishnu Ks 2ea53a347a import: Support importing realm icon and logo.
Fixes #11216
2020-02-03 14:09:05 -08:00
Vishnu Ks af3a37b58b upload: Refactor out realm_avatar_and_logo_path function. 2020-02-03 14:09:05 -08:00
Tim Abbott df110e8ff9 test_fixtures: Note populate_db depends on server_initialization.py.
This should ensure that folks rebasing past this commit from an older
database model get their database rebuilt in the way that will
match the test_subs.py query count of 40.
2020-02-03 10:38:04 -08:00
Ryan Rehman 3dc7d60ffe muting: Record DateTime when a Topic is muted.
This includes the necessary migration to add
the date_muted field to the MutedTopic class
and populates it with a hard coded value.
2020-02-02 20:49:53 -08:00
Mateusz Mandera bf89cf2b4b rate_limiter: Use ABC for defining the abstract class RateLimitedObject. 2020-02-02 19:15:13 -08:00
Mateusz Mandera cb71a6571e rate_limiter: Rename 'all' domain to 'api_by_user'. 2020-02-02 19:15:13 -08:00
Mateusz Mandera 06198af5b9 auth: Handle rate limiting in OurAuthenticationForm and user_settings.
These parts of the code should catch the RateLimited exception and
generate their own, apprioprate user-facing error message.
2020-02-02 19:15:13 -08:00
Mateusz Mandera 335b804510 exceptions: RateLimited shouldn't inherit from PermissionDenied.
We will want to raise RateLimited in authenticate() in rate limiting
code - Django's authenticate() mechanism catches PermissionDenied, which
we don't want for RateLimited. We want RateLimited to propagate to our
code that called the authenticate() function.
2020-02-02 19:15:00 -08:00
Mateusz Mandera a6a2d70320 rate_limiter: Handle multiple types of rate limiting in middleware.
As more types of rate limiting of requests are added, one request may
end up having various limits applied to it - and the middleware needs to
be able to handle that. We implement that through a set_response_headers
function, which sets the X-RateLimit-* headers in a sensible way based
on all the limits that were applied to the request.
2020-02-02 19:15:00 -08:00
Mateusz Mandera 4cc5d2464c rate_limiter: Expand support for different domains. 2020-02-02 19:15:00 -08:00
Tim Abbott 51706bdc3a stream: Deduplicate lists of stream/subscriptions fields.
While the result of this change doesn't completely do what we need, it
does remove a huge amount of duplicated lists of fields.  With a bit
more similar work, we should be able to eliminate a broad category of
potential bugs involving Stream and Subscription objects being
represented inconsistently in the API.

Work towards #13787.
2020-02-02 18:34:45 -08:00
Tim Abbott 238bc386cb actions: Deduplicate parts of get_web_public_subs.
This has the side of effect of making new fields we add to Stream be
automatically included, which will help maintain this code as we
upgrade it.

This commit adds is_web_public, history_public_to_subscribers, and
email_notifications fields to the dictionary.
2020-02-02 17:42:12 -08:00
Tim Abbott eac07698dd users: Add nocoverage tag for settings.SYSTEM_BOT_REALM conditional.
This is code for safety that should never happen and is likely
annoying to setup an automated test to verify.
2020-01-31 14:51:12 -08:00
Tim Abbott 5825a155cc users: Use format_user_row in events system as well.
This completes the deduplication of our logic for turning users into
dictionaries in the Zulip API.
2020-01-31 14:47:16 -08:00
akashaviator 20b8b29d11 users: Rewrite get_cross_realm_dicts to call format_user_row.
This modifies get_cross_realm_dicts in zerver.lib.users to call
format_user_row.  This is done to remove current and prevent future
inconsistencies between in the dictionary formats for get_raw_user_data
and get_cross_realm_dicts.

Implementation substantially rewritten by tabbott.

Fixes #13638.
2020-01-31 14:28:46 -08:00
akashaviator 7d06293ac0 refactor: Cleanup actions.py and events.py in zerver/lib.
This moves get_cross_realm_dicts (from zerver.lib.actions),
get_raw_user_data and get_custom_profile_field_values (from
zerver.lib.events) to zerver.lib.users.
2020-01-31 13:53:47 -08:00
Vishnu KS bd460af099 emails: Remove unecessary call to message_content_allowed_in_missedmessage_emails. 2020-01-31 12:29:58 -08:00
Vishnu KS 47e442e4a4 emails: Show proper message when email content is not shown. 2020-01-31 12:29:58 -08:00
akashaviator bd58e3397f events: Extract user_data function from get_raw_user_data.
This extracts the user_data inner function from get_raw_user_data as a
reusable function.  We intend to reuse it for cross-realm user dicts.
A few changes were made while extracting it:

* Renaming the UserProfile argument to acting_user, so we can do loops
  over a local user_profile variable.
* Moved it to zerver.lib.users, as that's a more appropriate home for
  this function formatting data on users.

* Simplified the calling convention for passing custom profile fields
  to reflect the fact that this function processes a single user (and
  is expected to be called in a loop).
2020-01-30 13:32:35 -08:00
Mateusz Mandera 8bd3752d13 email_mirror: Handle encoded attachment filenames. 2020-01-30 13:03:47 -08:00
Mateusz Mandera 49b76318c6 email_mirror: Extract handle_header_content function. 2020-01-30 13:03:47 -08:00