Commit Graph

36977 Commits

Author SHA1 Message Date
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Alex Vandiver b114eb2f10 puppet: Rename env-wal-e to env-wal-g.
It runs wal-g now, not wal-e; make its name respect that.
2020-06-11 15:52:43 -07:00
Alex Vandiver 4fe0444108 puppet: Install wal-g, not wal-e. 2020-06-11 15:52:43 -07:00
YashRE42 6506447bf1 navbar: Only set searchbox text when displaying the searchbox.
Previously, the navbar failed at managing the searchbox text state in
cases where, eg, the user performs navigation by browser history.

This commit resolves the issue by ensuring that the searchbox text is
only (and always) set when the searchbox is made visible, and as such
there is no "state" to manage and we will always display the correct
text.

It also adds a test in `search_legacy.js` to make sure that the search
text is placed as intended.

Fixes: #14771.
2020-06-11 15:49:12 -07:00
YashRE42 939f040674 navbar: Extract `reset_searchbox_text` in tab_bar.
The navbar currently fails at managing the searchbox text state in
cases where, eg, the user performs navigation by browser history.
This is a prep commit that will help resolve the bug.
2020-06-11 15:49:12 -07:00
Anders Kaseorg 0e5946ee5a python: Add noqa comments for the specific star imports we allow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 15:36:43 -07:00
Anders Kaseorg 491fb3c3cf urls: Avoid bare tuples.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 15:36:43 -07:00
Anders Kaseorg bf7c421351 test_signup: Remove spurious bare tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 15:36:43 -07:00
Anders Kaseorg aca065e1c1 logging_handlers: Remove spurious bare tuples.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 15:36:43 -07:00
Anders Kaseorg 03e147d5e1 python: Replace NamedTuple with dataclass.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 15:19:31 -07:00
Alex Vandiver 39d6185ce7 puppet: Remove python-dateutil requirement from pg_backup_and_purge.
1f565a9f41 removed the `package` lines which install
`python-dateutil`, but not the line in `puppet_ops` that reference it;
as such, Puppet manifests in puppet_ops fail to compile.

Remove the stale reference to `python-dateutil`, which is unnecessary
since the code is python3, not python2.
2020-06-11 14:28:55 -07:00
Graham Bleaney 461d5b1a3e pysa: Introduce sanitizers, models, and inline marking safe.
This commit adds three `.pysa` model files: `false_positives.pysa`
for ruling out false positive flows with `Sanitize` annotations,
`req_lib.pysa` for educating pysa about Zulip's `REQ()` pattern for
extracting user input, and `redirects.pysa` for capturing the risk
of open redirects within Zulip code. Additionally, this commit
introduces `mark_sanitized`, an identity function which can be used
to selectively clear taint in cases where `Sanitize` models will not
work. This commit also puts `mark_sanitized` to work removing known
false postive flows.
2020-06-11 12:57:49 -07:00
Graham Bleaney 89131bfcbb pysa: Add basic .pyre_configuration and taint.config for pysa.
This commit adds the initial config files needed to be able to run
Pysa on Zulip code.
2020-06-11 12:57:49 -07:00
Graham Bleaney 56345f98cb registration: Incorporate redirect URL building helper function.
This commit adds 'add_query_to_redirect_url' to one additional
function which had not yet been written when
'add_query_to_redirect_url' was introduced. This helper centralizes
URL manipulation for redirects, making it easier to add Pysa
sanitization in subsequent commits.
2020-06-11 12:57:49 -07:00
Anders Kaseorg ca4357fd64 python: Use standard NoReturn (Python ≥ 3.6).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 12:56:52 -07:00
Mateusz Mandera fbc96d56d5 sharding: Fix permissions on the nginx_sharding.conf file.
The zulip user needs to be able to read the file, when running the
backup tool.
We put root:root as owner on other nginx config files, so it's probably
correct to keep the ownership as it is, and set the mode to 0644.
2020-06-11 12:56:06 -07:00
Vishnu KS bc57d9b611 i18n: Don't hardcode zulip.com URLs in translation tags. 2020-06-11 11:09:42 -07:00
Steve Howell 5c6fb8dbbf code cleanup: Avoid shadowing the tab_bar global.
I tried to make the most minimal change here
possible, since I don't really know this module
well.  Possible alternatives were:

    * $('#tab_bar') everywhere
    * elem
    * tab_bar_elem

I don't feel strongly.

Long term I believe we intend to change the name
of this module to something more like `navbar.js`???
2020-06-11 11:05:06 -07:00
Steve Howell 863660281e code cleanup: Use exports for internal references.
When we call functions inside our own modules that use
the `window.foo = exports` pattern, we have always had
a pretty strong preference to call `exports.internal_function`
instead of `foo.internal_functions`.

The stragglers here weren't violating this convention
for any intentional reason.  Some of the places here
probably were part of code moves where somebody
(probably me) moved functions into the modules to avoid
unnecessary indirection, and I missed a spot where I
could change from `presence` to `exports` (or whatever).

And other places are probably just kinda arbitrary
decisions by the original developer, and we just haven't
bothered to clean it up until now.
2020-06-11 11:05:06 -07:00
Kartik Srivastava 8c39ddfd28 api: Add GET /users/{user_id}/subscription/{stream_id} endpoint.
This new endpoint returns a 'user' dictionary which, as of now,
contains a single key 'is_subscribed' with a boolean value that
represents whether the user with the given 'user_id' is subscribed
to the stream with the given 'stream_id'.

Fixes #14966.
2020-06-10 17:59:14 -07:00
Steve Howell d5cc29755e typing: Accept only user_ids in typing endpoint.
The only clients that should use the typing
indicators endpoint are our internal clients,
and they should send a JSON-formatted list
of user_ids.

We now enforce this, which removes some
complexity surrounding legacy ways of sending
users, such as emails and comma-delimited
strings of user_ids.

There may be a very tiny number of mobile
clients that still use the old emails API.

This won't have any user-facing effect on
the mobile users themselves, but if you type
a message to your friend on an old mobile
app, the friend will no longer see typing
indicators.

Also, the mobile team may see some errors
in their Sentry logs from the server rejecting
posts from the old mobile clients.

The error messages we report here are a bit
more generic, since we now just use REQ
to do validation with this code:

    validator=check_list(check_int)

This also allows us to remove a test hack
related to the API documentation.  (We changed
the docs to reflect the modern API in an
earlier commit, but the tests couldn't be
fixed while we still had the more complex
semantics for the "to" parameter.)
2020-06-10 17:50:32 -07:00
sahil839 81c28c1d3e realm: Allow only organization owners to deactivate a realm.
We now allow only organization owners to deactivate a realm.

'require_realm_owner' decorator has been added for this purpose.
2020-06-10 17:33:02 -07:00
sahil839 86b52ef7bf users: Owners can only be deactivated by other organization owners.
This commit adds restriction on deactivation of organization owners.
Only owners can deactivate other organization owners.
2020-06-10 17:33:02 -07:00
jagansivam28 b40597cf3e settings_org: Combine upload_realm_logo and upload_realm_icon function.
This combines `upload_realm_logo` and `upload_realm_icon` into single
function called `upload_realm_logo_or_icon`.  The functions wer near
duplicates.

Additional refactoring should be able to deduplicate the logic further.

Part of #14730.
2020-06-10 17:05:29 -07:00
jagansivam28 1c3485647f realm logo: Refactor upload_widgets.js.
We can remove a {{theme_mode}}-settings class on the upload button
and access day/night from `.closest("realm-logo-section")`
so that only the outer ID differs between the two widgets.

Part of #14730.
2020-06-10 17:02:06 -07:00
Vinit Singh 8b2098f34d refactor: Add css nesting to #out-of-view-notification. 2020-06-10 17:00:39 -07:00
Vinit Singh dc96b8a23d drafts: Show drafts count on draft button.
This change displays the number of currently saved drafts
on the draft button like "Drafts (2)".

Patially resolves #5591.
2020-06-10 17:00:39 -07:00
orientor e989c87a18 openapi: Fix response validation bug.
Currently there are no checks in validate_against_openapi_schema
to check whether the `content` it received actually matched with
the `response code`. For example during testing if a certain endpoint
was returning 400 but it was expected to return 200, then it would
pass schema validation as it would only have `msg` and `result` keys.
Add this validation and fix the wrong response returning points.
2020-06-10 16:45:36 -07:00
orientor 03ef5e0b31 openapi: Add response example validation.
Add test to validate example responses in zulip.yaml. Also change
zulip.yaml for some wrong examples or for cases which were not
covered by `test-api`. Also enhance `validate_against_openapi_schema`.
2020-06-10 15:45:50 -07:00
Anders Kaseorg 67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg 6480deaf27 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, with more
restrictions patched out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 14:48:09 -07:00
Hemanth V. Alluri 7b2255597c outgoing-webhook: Fix check for empty content.
Prevent `JsonableError(_("Missing content"))` from
ever being triggered.

That error wasn't handle by anything, and thus just threw a 500, as
it's not a response to an HTTP request.

The right fix is to adjust the caller to ban the empty string in
content (or content that strips to the empty string).

Closes #15145.
2020-06-10 14:46:04 -07:00
Dinesh 0c45e0118f tests: Clear remove/remock approach for ACCESS_TOKEN_URL endpoint.
ACCESS_TOKEN_URL works a different for apple authentication, so,
we removed and remocked the ACCESS_TOKEN_URL mock in
`register_extra_endpoints` override of apple auth test class.
It is cleaner to have it as generic feature of `social_auth_test`.

So, this commit adds a function that returns token_data_dict that
we had earlier and is called in the ACCESS_TOKEN_URL mock.
This function is overriden in apple auth test class to generate
payload of the format that apple auth expects.

Thanks to Mateusz Mandera for the simple idea.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-06-10 14:38:49 -07:00
Tim Abbott 5fd7b519b7 templates: Add missing include file for owner-only.md. 2020-06-10 14:35:18 -07:00
YashRE42 4f6377d493 navbar: Correctly recolor navbar icon when stream is renamed.
Previously, renaming the stream would cause `colorize_tab_bar` to fail
because the search filter would provide it the old stream name and the
look up for the stream color would fail.

A quirk of how this system currently works makes it so that even
though the filter is set to the old stream name (and so becomes
inconsistent with the stream_data state) the `_stream_params` object
is maintained to be consistent with stream_data and as such can be
relied upon to find the correct color of the icon.
2020-06-10 14:25:18 -07:00
YashRE42 e389129f34 subs: Call `filter.fix_stream_params` when updating stream name.
Previously the navbar did not live update the stream name correctly.

The correct behaviour was blocked on the `_stream_name` prop on the
filter object. The original purpose for maintaining this prop was
convenience, to reduce calls to `subs`, however, it would become
inconsistent with the value from `subs` on stream rename.

In this commit we add a call to `filter.fix_stream_params` in
`subs.update_stream_name`. This fixes live rerendering in the navbar,
despite the fact that searchbox in the nav (and the filter object via
`filter.operands("stream")[0]`) will still have the old name.

This is a slightly hacky way of masking some of the problems in the
Filter object. However, it should make do until we migrate to a stream
ID based state there.

Fixes: #14728.
2020-06-10 14:25:18 -07:00
YashRE42 96e272081d navbar: Add globe icon to navbar for web public streams.
Prior to this commit, the navbar didn't display web public streams as
any different from ordinary streams.

Now, we show a globe icon for web public streams. This commit also
adds a node test for the same.

We also modified the navbar behaviour table, which is the following
dropbox paper:
https://paper.dropbox.com/doc/Navbar-behavior-table--A0sp57z~7R5PeHuxHL__Gn5ZAg-cNOGtu7kSdtnKBizKXJge
2020-06-10 14:25:18 -07:00
YashRE42 bb6ce47c33 navbar: Live update icon for stream privacy changes.
Prior to this commit the icon in the navbar didn't live update to
reflect changes in stream privacy.

Here, we add a call to `tab_bar.render_title_area` in
`subs.update_stream_privacy()`, to enable live updates on the icon.

Fixes #14728.
2020-06-10 14:25:18 -07:00
YashRE42 d496304caa subs: Call `filter.fix_stream_params` when updating stream privacy.
The navbar currently does not live update the stream icon correctly
for changes in stream privacy.

One place where the correct behaviour gets blocked is on the
`_is_stream_private` prop in the filter object. We keep props such as
this for convenience, to reduce calls to `subs`, however, this prop
becomes inconsistent with the value we get from `subs` when the stream
privacy is updated.

In this commit we add a call to `filter.fix_stream_params` in
`subs.update_stream_privacy`. This change does not fix the live
rerendering in the navbar because we don't call redraw yet, but
it's a prep commit to towards that goal
2020-06-10 14:25:18 -07:00
YashRE42 35c8dcb599 filter: Extract `get_stream_params` and `fix_stream_params`.
The navbar currently does not live update the stream name or the icon
for stream privacy correctly.

One place where the correct behaviour gets blocked is on the
`_stream_name` and `_is_stream_private` props on the filter object.
We keep these props for convenience, to reduce calls to `subs`,
however, these props become inconsistent with the values from `subs`
when the stream is renamed or stream privacy is changed.

This refactor extracts out `get_stream_params` and `fix_stream_params`
methods as a prep commit towards correcting the problem, but does not
make any behavioural changes.

This is a prep commit for solving #14728.
2020-06-10 14:25:18 -07:00
YashRE42 b292c30d6a subs: Move comment about live update for stream names in navbar.
A comment about the difficulties relating to live updating stream
names in the navbar was incorrectly placed within the function for
live updating stream descriptions in
023187b3f1.

This moves the comment to the right place.
2020-06-10 14:25:18 -07:00
Tim Abbott f0d8f60b66 help: Add basic documentation of organization owners. 2020-06-10 14:07:46 -07:00
Tim Abbott 94d0c330e4 management: Display organization owners in show_admins. 2020-06-10 14:07:38 -07:00
sahil839 22a188f12e migration: Add migration to set all realm admins as realm owners.
This migration is added to set all the realm admins as realm owners
and also creates RealmAuditLog entries for modified users.
2020-06-10 14:07:38 -07:00
sahil839 5cc937f670 events: Add 'is_owner' to page_params and raw_users dict of page_params.
This commit adds 'is_owner' field to page_params and also to the
raw_users dict of page_params.
2020-06-10 14:07:38 -07:00
sahil839 6d667dbe53 realm owner: Add ability to change realm owner status of user.
This commit adds some basic checks while adding or removing
realm owner status of a user and adds code to change owner
status of a user using update_user_backend.

This also adds restriction on removing owner status of the
last owner of realm. This restriction was previously on
revoking admin status, but as we have added a more privileged
role of realm owner, we now have this restriction on owner
instead of admin.

We need to apply that restriction both in the role change code path
and the deactivate code path.
2020-06-10 12:49:37 -07:00
sahil839 2c8c641556 realm owner: Make first user as realm owner on realm creation.
This commit sets the role of the user creating the realm as
realm owner after the realm is created.

Previously, the role of user creating the realm was set as admin.
But now we want it to be owner because owners have the highest
privilege level.
2020-06-10 12:44:52 -07:00
Clara Dantas 451ea753a2 settings_users: Simplify get_item function. 2020-06-10 12:14:58 -07:00
Tim Abbott 6f159ce5c5 templates: Clean up wording around topic moves.
This switches us to cleaner, simpler labels for moving topics; while
also avoiding claiming that we're only allowing moving topics to other
streams.
2020-06-10 12:12:17 -07:00
Tim Abbott 26083e4b10 popovers: Improve label for choosing colors. 2020-06-10 09:55:51 -07:00