Commit Graph

166 Commits

Author SHA1 Message Date
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg 04919528e7 models: Parameterize .extra(where=["… IN …"]) with tuple/list adapter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 21:12:43 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
sahil839 9ef1c5b1a6 users: Add is_owner field to user objects returned by get endpoints.
This commit adds 'is_owner' field to the user object returned by
'/users', 'users/{user_id}', and '/users/me' endpoints.
2020-06-01 15:33:51 -07:00
sahil839 169bd3082a users: Modify format_user_row to set is_admin as True for owners also. 2020-06-01 15:14:16 -07:00
sahil839 2ab6767b73 events: Update person dict in event for do_change_user_role to send role.
This commit changes the person dict in event sent by do_change_user_role
to send role instead of is_admin or is_guest.

This makes things much more straightforward for our upcoming primary
owners feature.
2020-05-31 17:22:50 -07:00
clarammdantas 7e9024a39c popovers.js: Add version to user avatar request.
When a user changes its avatar image, the user's avatar in popovers
wasn't being correctly updated, because of browser caching of the
avatar image.  We added a version on the request to get the image in
the same format we use elsewhere, so the browser knows when to use the
cached image or to make a new request to the server.

Edited by Tim to preserve/fix sort orders in some tests, and update
zulip_feature_level.

Fixes: #14290
2020-05-12 11:09:01 -07:00
Abhishek-Balaji 43e39718c4 user_name: Prevent users from setting name ending with |number.
We've had bugs in the past where users with a name in the format
"Alice|999" would confuse our markdown rendering or typeahead.  While
that's a fully solvable problem, there's no real use case for that, so
it's probably simpler to just prevent users from setting their name
that way.

Fixes #13923.
2020-04-30 15:59:12 -07:00
Anders Kaseorg fead14951c python: Convert assignment type annotations to Python 3.6 style.
This commit was split by tabbott; this piece covers the vast majority
of files in Zulip, but excludes scripts/, tools/, and puppet/ to help
ensure we at least show the right error messages for Xenial systems.

We can likely further refine the remaining pieces with some testing.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-    invoiced_through: Optional[LicenseLedger] = models.ForeignKey(
+    invoiced_through: Optional["LicenseLedger"] = models.ForeignKey(

-_apns_client: Optional[APNsClient] = None
+_apns_client: Optional["APNsClient"] = None

-    notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
-    signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)

-    author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)
+    author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)

-    bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)
+    bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)

-    default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
-    default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)

-descriptors_by_handler_id: Dict[int, ClientDescriptor] = {}
+descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {}

-worker_classes: Dict[str, Type[QueueProcessingWorker]] = {}
-queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {}
+worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {}
+queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {}

-AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None
+AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 11:02:32 -07:00
Steve Howell 2ff41bf9e5 /json/users: Use field.realm for realm lookup.
This avoids an unnecessary join to UserProfile.

To verify this, you can do `print(queries)` in the
`test_get_custom_profile_fields_from_api` test.  It's
kinda noisy, so I excerpted them below...

Before:

    SELECT ...
    FROM "zerver_customprofilefieldvalue"
    INNER JOIN "zerver_userprofile" ON ("zerver_customprofilefieldvalue"."user_profile_id" = "zerver_userprofile"."id")
    INNER JOIN "zerver_customprofilefield" ON ("zerver_customprofilefieldvalue"."field_id" = "zerver_customprofilefield"."id")
    WHERE "zerver_userprofile"."realm_id" = 2

After:

    SELECT ...
    FROM "zerver_customprofilefieldvalue"
    INNER JOIN "zerver_customprofilefield" ON ("zerver_customprofilefieldvalue"."field_id" = "zerver_customprofilefield"."id")
    WHERE "zerver_customprofilefield"."realm_id" = 2'

I don't have any way to measure the two queries with
realistic data, but I would assume the second
query is significantly faster on most of our instances,
since CustomProfileField should be tiny.
2020-02-09 22:04:02 -08:00
Steve Howell 01f180d042 minor: Remove unused line of code in get_raw_user_data().
The line removed here is a noop, as both sides of the
immediately following conditional reassign the
same variable.

This harmless cruft was the result of the recent commit
1ae5964ab8, which added
support for single-user GETs.
2020-02-09 22:04:02 -08:00
akashaviator 1ae5964ab8 api: Add an api endpoint for GET /users/{id}
This adds a new API endpoint for querying basic data on a single other
user in the organization, reusing the existing infrastructure (and
view function!) for getting data on all users in an organization.

Fixes #12277.
2020-02-07 10:36:31 -08:00
Tim Abbott e39840c705 users: Add read-only mode for access_user_by_id.
We've be using this in the upcoming GET /users/{id} method.
2020-02-07 10:36:31 -08:00
Tim Abbott aa9286a1f9 users: Move query into caller of get_custom_profile_field_values.
This will be useful for supporting a smaller query for a single user.
2020-02-07 10:36:31 -08:00
Tim Abbott 79e5dd1374 users: Rename get_raw_user_data user parameter to acting_user.
This is for improved clarity as we extend this function to take
multiple user objects.
2020-02-07 10:36:31 -08:00
Tim Abbott 7c0a98754a home: Refactor logic for show_invites and show_add_streams. 2020-02-05 16:05:02 -08:00
Tim Abbott eac07698dd users: Add nocoverage tag for settings.SYSTEM_BOT_REALM conditional.
This is code for safety that should never happen and is likely
annoying to setup an automated test to verify.
2020-01-31 14:51:12 -08:00
Tim Abbott 5825a155cc users: Use format_user_row in events system as well.
This completes the deduplication of our logic for turning users into
dictionaries in the Zulip API.
2020-01-31 14:47:16 -08:00
akashaviator 20b8b29d11 users: Rewrite get_cross_realm_dicts to call format_user_row.
This modifies get_cross_realm_dicts in zerver.lib.users to call
format_user_row.  This is done to remove current and prevent future
inconsistencies between in the dictionary formats for get_raw_user_data
and get_cross_realm_dicts.

Implementation substantially rewritten by tabbott.

Fixes #13638.
2020-01-31 14:28:46 -08:00
akashaviator 7d06293ac0 refactor: Cleanup actions.py and events.py in zerver/lib.
This moves get_cross_realm_dicts (from zerver.lib.actions),
get_raw_user_data and get_custom_profile_field_values (from
zerver.lib.events) to zerver.lib.users.
2020-01-31 13:53:47 -08:00
akashaviator bd58e3397f events: Extract user_data function from get_raw_user_data.
This extracts the user_data inner function from get_raw_user_data as a
reusable function.  We intend to reuse it for cross-realm user dicts.
A few changes were made while extracting it:

* Renaming the UserProfile argument to acting_user, so we can do loops
  over a local user_profile variable.
* Moved it to zerver.lib.users, as that's a more appropriate home for
  this function formatting data on users.

* Simplified the calling convention for passing custom profile fields
  to reflect the fact that this function processes a single user (and
  is expected to be called in a loop).
2020-01-30 13:32:35 -08:00
Abhishek-Balaji 434e8d3104 home: Extract compute_show_invites_and_add_streams.
This extracts a function for computing show_invites and
show_add_streams, for better readability and testability.

This commit was substantially cleaned up by tabbott.
2020-01-25 23:41:08 -08:00
Matheus Melo 21ed834101 decorator: Extract OrganizationAdministratorRequired common exception.
This eliminates significant code duplication of error messages for
situations where an organization administrator is required.
2019-11-18 15:10:56 -08:00
Hemanth V. Alluri d73a37726d bots: Allow incoming webhook bots to be configured via /bots.
Without disturbing the flow of the existing code for configuring
embedded bots too much, we now use the config_options feature to
allow incoming webhook type bot to be configured via. the "/bots"
endpoint of the API.
2019-08-20 17:00:48 -07:00
Hemanth V. Alluri 94c351ead4 bot_config: Have check_valid_bot_config also take the bot_type.
This is a prep commit to allow us to validate user provided bot
config data using the same function for incoming webhook type
bots alongside embedded bots (as opposed to creating a new
function just for incoming webhook bots).
2019-08-20 16:44:56 -07:00
Mateusz Mandera cb2c9b04b3 generic_bulk_cached_fetch: Only call query_function if necessary. 2019-08-15 17:14:02 -07:00
Tim Abbott 27a0e307b6 cache: Fix typing for generic_bulk_cached_fetch.
The typing for generic_bulk_cached_fetch is complicated, and was
recorded incorrectly previously for the case where a cache_transformer
function is required.  We fix this by adding the new CacheItemT, and
additionally add comments explaining what's going on with these types
for future reference.

Thanks to Mateusz Mandera for raising this issue.
2019-08-14 11:00:40 -07:00
Conner Bondurant c25dcf048d models: Enforce stricter requirements on the full_name field.
This changes the requirements for UserProfile to disallow some
additional characters, with the overall goal of being able to use
formataddr in send_mail.py.

We don't need to be particularly careful in the database migration,
because user full_names are not required to be unique.
2019-07-22 18:13:34 -07:00
Anders Kaseorg 9a9de156c3 lint: Fix calls to _() on computed strings.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:23:03 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00
Anders Kaseorg f0ecb93515 zerver core: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:41:24 -08:00
Harshit Bansal 6777b94d41 lib: Extract `validate_user_custom_profile_field()`. 2019-01-29 16:01:30 -08:00
Vishnu Ks e5f7d65231 signup: Set oldest account as default option in import settings.
Fixes: #11018
2018-12-29 15:01:09 -08:00
Tim Abbott 14bfa74069 registration: Enable copying profile settings in production.
Now that we've styled this feature properly, this makes it possible to
copy various user-preferences type profile data in production when
making a new account with the same email address as an existing
account.
2018-12-10 16:55:07 -08:00
Tim Abbott e603237010 email: Convert accounts code to use delivery_email.
A key part of this is the new helper, get_user_by_delivery_email.  Its
verbose name is important for clarity; it should help avoid blind
copy-pasting of get_user (which we'll also want to rename).
Unfortunately, it requires detailed understanding of the context to
figure out which one to use; each is used in about half of call sites.

Another important note is that this PR doesn't migrate get_user calls
in the tests except where not doing so would cause the tests to fail.
This probably deserves a follow-up refactor to avoid bugs here.
2018-12-06 16:21:38 -08:00
Yashashvi Dave 4ceb4b607f zerver/lib/users.py: Extract func `validate_user_custom_profile_data`.
Extract function `validate_user_custom_profile_data` to validate
user's custom profile field values according to field type.
2018-10-31 15:36:44 -07:00
Tim Abbott 462b9c80c0 bots: Add basic documentation for duplicate bot names feature. 2018-10-24 17:01:34 -07:00
Steve Howell 551fc7f165 bots: Prevent bots from having duplicate full names.
Bots are not allowed to use the same name as
other users in the realm (either bot or human).

This is kind of a big commit, but I wanted to
combine the post/patch (aka add/edit) checks
into one commit, since it's a change in policy
that affects both codepaths.

A lot of the noise is in tests.  We had good
coverage on the previous code, including some places
like event testing where we were expediently
not bothering to use different names for
different bots in some longer tests.  And then
of course I test some new scenarios that are relevant
with the new policy.

There are two new functions:

    check_bot_name_available:
        very simple Django query

    check_change_bot_full_name:
        this diverges from the 3-line
        check_change_full_name, where the latter
        is still used for the "humans" use case

And then we just call those in appropriate places.

Note that there is still a loophole here
where you can get two bots with the same
name if you reactivate a bot named Fred
that was inactive when the second bot named
Fred was created.  Also, we don't attempt
to fix historical data.  So this commit
shouldn't be considered any kind of lockdown,
it's just meant to help people from
inadvertently creating two bots of the same
name where they don't intend to.  For more
context, we are continuing to allow two
human users in the same realm to have the
same full name, and our code should generally
be tolerant of that possibility.  (A good
example is our new mention syntax, which disambiguates
same-named people using ids.)

It's also worth noting that our web app client
doesn't try to scrub full_name from its payload in
situations where the user has actually only modified other
fields in the "Edit bot" UI.  Starting here
we just handle this on the server, since it's
easy to fix there, and even if we fixed it in the web
app, there's no guarantee that other clients won't be
just as brute force.  It wasn't exactly broken before,
but we'd needlessly write rows to audit tables.

Fixes #10509
2018-10-24 16:59:57 -07:00
Yago González f6219745de users: Get all API keys via wrapper method.
Now reading API keys from a user is done with the get_api_key wrapper
method, rather than directly fetching it from the user object.

Also, every place where an action should be done for each API key is now
using get_all_api_keys. This method returns for the moment a single-item
list, containing the specified user's API key.

This commit is the first step towards allowing users have multiple API
keys.
2018-08-08 16:35:17 -07:00
Vishnu Ks 403f254557 signup: Create get_accounts_for_email function. 2018-06-19 11:25:23 -07:00
Yashashvi Dave 5145b24635 users: Replace duplication with generic func to validate user id.
This adds a common function `access_user_by_id` to access user id
within same realm, complete with a full suite of unit tests.

Tweaked by tabbott to make the test much more readable.
2018-06-05 11:13:13 -07:00
Tim Abbott 0d84eb0a8b mypy: Clean zerver/views/users.py for strict-optional. 2018-06-01 08:48:17 -07:00
Tim Abbott 0a0ae5e703 users: Move add_service to zerver.lib.users so it is reusable. 2018-06-01 08:48:17 -07:00
Yashashvi Dave 47aaf4e20a users: Replace duplication with generic func to validate bot id.
This adds a common function `access_bot_by_id` to access bot id within
same realm.  It probably fixes some corner case bugs where we weren't
checking for deactivated bots when regenerating API keys.
2018-05-29 15:47:27 -07:00
Aditya Bansal a68376e2ba zerver/lib: Change use of typing.Text to str. 2018-05-12 15:22:39 -07:00
Tim Abbott c06565d909 users: Improve testing for user_ids_to_users. 2018-04-04 16:31:30 -07:00
novokrest 807a6ccf2c users.py: Implement user_ids_to_users() by generic_bulk_cached_fetch().
Optimize users.user_ids_to_users() implementation
by using generic_bulk_cached_fetch() to obtain user profiles
2018-04-04 16:24:55 -07:00
neiljp (Neil Pilgrim) d741e0ea01 mypy: Introduce query variable in lib/users.py for clarity.
This also avoids mypy showsing an error.
2018-03-23 11:32:00 -07:00
Shubham Dhama 777b6de689 org settings: Add setting to prevent users from adding bots.
Fixes: #7908.
2018-03-09 13:21:55 -08:00
Robert Hönig 649e76e932 Display error when creating embedded bot with incorrect config data.
"incorrect" here means rejected by a bot's validate_config() method.
A common scenario for this is validating API keys before the bot is
created. If validate_config() fails, the bot will not be created.
2018-03-08 15:05:42 -08:00
Alena Volkova 45f0c76c44 settings: Limit the creation of generic bots.
This commit adds a setting to limit creation of generic bots
to admins for realms that want that restriction.  (Generic
bots, apart from being considered spammy on some realms,
have less locked down permissions than webhook bots).

Fixes #7066.
2018-01-02 18:12:22 -05:00
Tim Abbott 25fd4c5508 bulk_get_users: Edit the cache keys to make them more unique.
While at this point I was to rewrite this function, this at least
plugs the issues for now.
2017-11-27 14:41:31 -08:00
Tim Abbott 646ba5b9e5 bulk_get_users: Fix issues with users in multiple realms.
The previous implementation had a subtle caching bug: because it was
sharing its cache with the `get_user_profile_by_email` cache, if a
user happened to have an email in that cache, we'd return it, even
though that user didn't match `base_query`.

This causes `get_cross_realm_users` to no longer have a problematic
caching bug.
2017-11-27 14:34:45 -08:00
rht 3f4bf2d22f zerver/lib: Use python 3 syntax for typing.
Extracted from a larger commit by tabbott because these changes will
not create significant merge conflicts.
2017-11-21 20:56:40 -08:00
Tim Abbott f02e5b90f6 cross_realm: Use bulk_get_users to fix handling of missing users.
This fixes a regression in ae5ba7f4fd,
where Zulip would 500 if the newly added system bots didn't exist on
the server.

This also fixes a moderate size performance problem where we'd fetch 5
users from memcached or the database in a loop.
2017-11-15 21:24:51 -08:00
Tim Abbott c7a975e4df users: Move check_change_full_name to actions.py.
This avoids an import loop in the next commit, and better matches our
usual code structure.
2017-11-15 17:39:09 -08:00
Umair Khan 1bbe84af49 user-groups: Add create API endpoint.
Significantly modified by tabbott for better security structure.
2017-11-09 17:26:14 -08:00
rht f43e54d352 zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
vaibhav 691aff55a3 bots: Add UI to select Slack compatible interface for webhooks.
interface_type select menu will be used to choose the interface
for outgoing webhooks. It will be displayed only when the selected
bot type is OUTGOING WEBHOOK type. The default value is GENERIC
interface type (1).
2017-07-28 16:22:55 -07:00
Sampriti Panda 5dc053d6fb bots: Add validation to add_bot_backend to prevent empty short names
Fixes #5487
2017-06-21 10:11:08 -04:00
Abhijeet Kaur 60ff82ed7c bots: Add UI for creating different types of bot.
Add 'Type of bot' option for bots by adding dropdown option in
settings->"Your bots".  For now, this allows creating incoming webhook
bots in addition to default bots.

This will enable users to add a bot as an incoming webhook
(in addition to add full-featured bots).

With various minor tweaks and cleanups by tabbott.

Fixes #2186.
2017-06-06 21:11:22 -07:00
Maxim Averin 685fb16c39 Switch change_full_name to use RealmAuditLog.
This requires adding an `acting_user` parameter to the
`do_change_bot_owner` function.
2017-05-29 15:22:08 -07:00
Tim Abbott 13a37f74a1 users: Ban names shorter than 3 characters.
The empty string is not a reasonable name.
2017-05-11 19:21:51 -07:00
Tim Abbott 56cecc4891 users: Verify full names explicitly in user creation.
This fixes an issue where users could be created with an invalid name
(introduced only a couple commits ago when we added character set
restrictions).
2017-02-07 19:54:30 -08:00
Tim Abbott 2283b5fc91 users: Consolidate name change enforcement logic.
This has the side effect of fixing an issue where one could edit a
bot to have an invalid name.
2017-02-07 19:45:21 -08:00