Commit Graph

36966 Commits

Author SHA1 Message Date
Graham Bleaney 461d5b1a3e pysa: Introduce sanitizers, models, and inline marking safe.
This commit adds three `.pysa` model files: `false_positives.pysa`
for ruling out false positive flows with `Sanitize` annotations,
`req_lib.pysa` for educating pysa about Zulip's `REQ()` pattern for
extracting user input, and `redirects.pysa` for capturing the risk
of open redirects within Zulip code. Additionally, this commit
introduces `mark_sanitized`, an identity function which can be used
to selectively clear taint in cases where `Sanitize` models will not
work. This commit also puts `mark_sanitized` to work removing known
false postive flows.
2020-06-11 12:57:49 -07:00
Graham Bleaney 89131bfcbb pysa: Add basic .pyre_configuration and taint.config for pysa.
This commit adds the initial config files needed to be able to run
Pysa on Zulip code.
2020-06-11 12:57:49 -07:00
Graham Bleaney 56345f98cb registration: Incorporate redirect URL building helper function.
This commit adds 'add_query_to_redirect_url' to one additional
function which had not yet been written when
'add_query_to_redirect_url' was introduced. This helper centralizes
URL manipulation for redirects, making it easier to add Pysa
sanitization in subsequent commits.
2020-06-11 12:57:49 -07:00
Anders Kaseorg ca4357fd64 python: Use standard NoReturn (Python ≥ 3.6).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 12:56:52 -07:00
Mateusz Mandera fbc96d56d5 sharding: Fix permissions on the nginx_sharding.conf file.
The zulip user needs to be able to read the file, when running the
backup tool.
We put root:root as owner on other nginx config files, so it's probably
correct to keep the ownership as it is, and set the mode to 0644.
2020-06-11 12:56:06 -07:00
Vishnu KS bc57d9b611 i18n: Don't hardcode zulip.com URLs in translation tags. 2020-06-11 11:09:42 -07:00
Steve Howell 5c6fb8dbbf code cleanup: Avoid shadowing the tab_bar global.
I tried to make the most minimal change here
possible, since I don't really know this module
well.  Possible alternatives were:

    * $('#tab_bar') everywhere
    * elem
    * tab_bar_elem

I don't feel strongly.

Long term I believe we intend to change the name
of this module to something more like `navbar.js`???
2020-06-11 11:05:06 -07:00
Steve Howell 863660281e code cleanup: Use exports for internal references.
When we call functions inside our own modules that use
the `window.foo = exports` pattern, we have always had
a pretty strong preference to call `exports.internal_function`
instead of `foo.internal_functions`.

The stragglers here weren't violating this convention
for any intentional reason.  Some of the places here
probably were part of code moves where somebody
(probably me) moved functions into the modules to avoid
unnecessary indirection, and I missed a spot where I
could change from `presence` to `exports` (or whatever).

And other places are probably just kinda arbitrary
decisions by the original developer, and we just haven't
bothered to clean it up until now.
2020-06-11 11:05:06 -07:00
Kartik Srivastava 8c39ddfd28 api: Add GET /users/{user_id}/subscription/{stream_id} endpoint.
This new endpoint returns a 'user' dictionary which, as of now,
contains a single key 'is_subscribed' with a boolean value that
represents whether the user with the given 'user_id' is subscribed
to the stream with the given 'stream_id'.

Fixes #14966.
2020-06-10 17:59:14 -07:00
Steve Howell d5cc29755e typing: Accept only user_ids in typing endpoint.
The only clients that should use the typing
indicators endpoint are our internal clients,
and they should send a JSON-formatted list
of user_ids.

We now enforce this, which removes some
complexity surrounding legacy ways of sending
users, such as emails and comma-delimited
strings of user_ids.

There may be a very tiny number of mobile
clients that still use the old emails API.

This won't have any user-facing effect on
the mobile users themselves, but if you type
a message to your friend on an old mobile
app, the friend will no longer see typing
indicators.

Also, the mobile team may see some errors
in their Sentry logs from the server rejecting
posts from the old mobile clients.

The error messages we report here are a bit
more generic, since we now just use REQ
to do validation with this code:

    validator=check_list(check_int)

This also allows us to remove a test hack
related to the API documentation.  (We changed
the docs to reflect the modern API in an
earlier commit, but the tests couldn't be
fixed while we still had the more complex
semantics for the "to" parameter.)
2020-06-10 17:50:32 -07:00
sahil839 81c28c1d3e realm: Allow only organization owners to deactivate a realm.
We now allow only organization owners to deactivate a realm.

'require_realm_owner' decorator has been added for this purpose.
2020-06-10 17:33:02 -07:00
sahil839 86b52ef7bf users: Owners can only be deactivated by other organization owners.
This commit adds restriction on deactivation of organization owners.
Only owners can deactivate other organization owners.
2020-06-10 17:33:02 -07:00
jagansivam28 b40597cf3e settings_org: Combine upload_realm_logo and upload_realm_icon function.
This combines `upload_realm_logo` and `upload_realm_icon` into single
function called `upload_realm_logo_or_icon`.  The functions wer near
duplicates.

Additional refactoring should be able to deduplicate the logic further.

Part of #14730.
2020-06-10 17:05:29 -07:00
jagansivam28 1c3485647f realm logo: Refactor upload_widgets.js.
We can remove a {{theme_mode}}-settings class on the upload button
and access day/night from `.closest("realm-logo-section")`
so that only the outer ID differs between the two widgets.

Part of #14730.
2020-06-10 17:02:06 -07:00
Vinit Singh 8b2098f34d refactor: Add css nesting to #out-of-view-notification. 2020-06-10 17:00:39 -07:00
Vinit Singh dc96b8a23d drafts: Show drafts count on draft button.
This change displays the number of currently saved drafts
on the draft button like "Drafts (2)".

Patially resolves #5591.
2020-06-10 17:00:39 -07:00
orientor e989c87a18 openapi: Fix response validation bug.
Currently there are no checks in validate_against_openapi_schema
to check whether the `content` it received actually matched with
the `response code`. For example during testing if a certain endpoint
was returning 400 but it was expected to return 200, then it would
pass schema validation as it would only have `msg` and `result` keys.
Add this validation and fix the wrong response returning points.
2020-06-10 16:45:36 -07:00
orientor 03ef5e0b31 openapi: Add response example validation.
Add test to validate example responses in zulip.yaml. Also change
zulip.yaml for some wrong examples or for cases which were not
covered by `test-api`. Also enhance `validate_against_openapi_schema`.
2020-06-10 15:45:50 -07:00
Anders Kaseorg 67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg 6480deaf27 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, with more
restrictions patched out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 14:48:09 -07:00
Hemanth V. Alluri 7b2255597c outgoing-webhook: Fix check for empty content.
Prevent `JsonableError(_("Missing content"))` from
ever being triggered.

That error wasn't handle by anything, and thus just threw a 500, as
it's not a response to an HTTP request.

The right fix is to adjust the caller to ban the empty string in
content (or content that strips to the empty string).

Closes #15145.
2020-06-10 14:46:04 -07:00
Dinesh 0c45e0118f tests: Clear remove/remock approach for ACCESS_TOKEN_URL endpoint.
ACCESS_TOKEN_URL works a different for apple authentication, so,
we removed and remocked the ACCESS_TOKEN_URL mock in
`register_extra_endpoints` override of apple auth test class.
It is cleaner to have it as generic feature of `social_auth_test`.

So, this commit adds a function that returns token_data_dict that
we had earlier and is called in the ACCESS_TOKEN_URL mock.
This function is overriden in apple auth test class to generate
payload of the format that apple auth expects.

Thanks to Mateusz Mandera for the simple idea.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-06-10 14:38:49 -07:00
Tim Abbott 5fd7b519b7 templates: Add missing include file for owner-only.md. 2020-06-10 14:35:18 -07:00
YashRE42 4f6377d493 navbar: Correctly recolor navbar icon when stream is renamed.
Previously, renaming the stream would cause `colorize_tab_bar` to fail
because the search filter would provide it the old stream name and the
look up for the stream color would fail.

A quirk of how this system currently works makes it so that even
though the filter is set to the old stream name (and so becomes
inconsistent with the stream_data state) the `_stream_params` object
is maintained to be consistent with stream_data and as such can be
relied upon to find the correct color of the icon.
2020-06-10 14:25:18 -07:00
YashRE42 e389129f34 subs: Call `filter.fix_stream_params` when updating stream name.
Previously the navbar did not live update the stream name correctly.

The correct behaviour was blocked on the `_stream_name` prop on the
filter object. The original purpose for maintaining this prop was
convenience, to reduce calls to `subs`, however, it would become
inconsistent with the value from `subs` on stream rename.

In this commit we add a call to `filter.fix_stream_params` in
`subs.update_stream_name`. This fixes live rerendering in the navbar,
despite the fact that searchbox in the nav (and the filter object via
`filter.operands("stream")[0]`) will still have the old name.

This is a slightly hacky way of masking some of the problems in the
Filter object. However, it should make do until we migrate to a stream
ID based state there.

Fixes: #14728.
2020-06-10 14:25:18 -07:00
YashRE42 96e272081d navbar: Add globe icon to navbar for web public streams.
Prior to this commit, the navbar didn't display web public streams as
any different from ordinary streams.

Now, we show a globe icon for web public streams. This commit also
adds a node test for the same.

We also modified the navbar behaviour table, which is the following
dropbox paper:
https://paper.dropbox.com/doc/Navbar-behavior-table--A0sp57z~7R5PeHuxHL__Gn5ZAg-cNOGtu7kSdtnKBizKXJge
2020-06-10 14:25:18 -07:00
YashRE42 bb6ce47c33 navbar: Live update icon for stream privacy changes.
Prior to this commit the icon in the navbar didn't live update to
reflect changes in stream privacy.

Here, we add a call to `tab_bar.render_title_area` in
`subs.update_stream_privacy()`, to enable live updates on the icon.

Fixes #14728.
2020-06-10 14:25:18 -07:00
YashRE42 d496304caa subs: Call `filter.fix_stream_params` when updating stream privacy.
The navbar currently does not live update the stream icon correctly
for changes in stream privacy.

One place where the correct behaviour gets blocked is on the
`_is_stream_private` prop in the filter object. We keep props such as
this for convenience, to reduce calls to `subs`, however, this prop
becomes inconsistent with the value we get from `subs` when the stream
privacy is updated.

In this commit we add a call to `filter.fix_stream_params` in
`subs.update_stream_privacy`. This change does not fix the live
rerendering in the navbar because we don't call redraw yet, but
it's a prep commit to towards that goal
2020-06-10 14:25:18 -07:00
YashRE42 35c8dcb599 filter: Extract `get_stream_params` and `fix_stream_params`.
The navbar currently does not live update the stream name or the icon
for stream privacy correctly.

One place where the correct behaviour gets blocked is on the
`_stream_name` and `_is_stream_private` props on the filter object.
We keep these props for convenience, to reduce calls to `subs`,
however, these props become inconsistent with the values from `subs`
when the stream is renamed or stream privacy is changed.

This refactor extracts out `get_stream_params` and `fix_stream_params`
methods as a prep commit towards correcting the problem, but does not
make any behavioural changes.

This is a prep commit for solving #14728.
2020-06-10 14:25:18 -07:00
YashRE42 b292c30d6a subs: Move comment about live update for stream names in navbar.
A comment about the difficulties relating to live updating stream
names in the navbar was incorrectly placed within the function for
live updating stream descriptions in
023187b3f1.

This moves the comment to the right place.
2020-06-10 14:25:18 -07:00
Tim Abbott f0d8f60b66 help: Add basic documentation of organization owners. 2020-06-10 14:07:46 -07:00
Tim Abbott 94d0c330e4 management: Display organization owners in show_admins. 2020-06-10 14:07:38 -07:00
sahil839 22a188f12e migration: Add migration to set all realm admins as realm owners.
This migration is added to set all the realm admins as realm owners
and also creates RealmAuditLog entries for modified users.
2020-06-10 14:07:38 -07:00
sahil839 5cc937f670 events: Add 'is_owner' to page_params and raw_users dict of page_params.
This commit adds 'is_owner' field to page_params and also to the
raw_users dict of page_params.
2020-06-10 14:07:38 -07:00
sahil839 6d667dbe53 realm owner: Add ability to change realm owner status of user.
This commit adds some basic checks while adding or removing
realm owner status of a user and adds code to change owner
status of a user using update_user_backend.

This also adds restriction on removing owner status of the
last owner of realm. This restriction was previously on
revoking admin status, but as we have added a more privileged
role of realm owner, we now have this restriction on owner
instead of admin.

We need to apply that restriction both in the role change code path
and the deactivate code path.
2020-06-10 12:49:37 -07:00
sahil839 2c8c641556 realm owner: Make first user as realm owner on realm creation.
This commit sets the role of the user creating the realm as
realm owner after the realm is created.

Previously, the role of user creating the realm was set as admin.
But now we want it to be owner because owners have the highest
privilege level.
2020-06-10 12:44:52 -07:00
Clara Dantas 451ea753a2 settings_users: Simplify get_item function. 2020-06-10 12:14:58 -07:00
Tim Abbott 6f159ce5c5 templates: Clean up wording around topic moves.
This switches us to cleaner, simpler labels for moving topics; while
also avoiding claiming that we're only allowing moving topics to other
streams.
2020-06-10 12:12:17 -07:00
Tim Abbott 26083e4b10 popovers: Improve label for choosing colors. 2020-06-10 09:55:51 -07:00
Aman Agrawal 7585776c76 stream_header: Fix broken header color and outflowing boundary.
The stream header was moved below the selection option and the
boundary selection input was overflowing, both were fixed via
this commit.
2020-06-10 09:50:49 -07:00
Anders Kaseorg 5837560a82 dev_settings: Use Python 2-compatible annotation.
This file is imported from zthumbor.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 02:11:55 -07:00
Anders Kaseorg 3de3386c21 Revert "dev_settings: Deduplicate DEPLOY_ROOT definition."
This reverts commit b8acd82b72.
2020-06-10 02:11:00 -07:00
Anders Kaseorg d3e8af4ad2 python: Replace silly uses of filter().
The test_management_commands use in particular was causing pickling
errors when the test failed, because Python 3 filter returns an
iterator, not a list.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 23:31:59 -07:00
Anders Kaseorg 5c9d56d2f7 get_safe_redirect_to: Remove incorrect allowed_hosts setting.
set(redirect_host) is a set of characters, so the only non-relative
redirects being allowed were to certain one-character hostnames, which
certainly isn’t what was intended.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:51:52 -07:00
Anders Kaseorg 3decbd1e23 lint: Remove settings exemption for possibly undefined star imports.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:29:50 -07:00
Anders Kaseorg 3a15e4b51d settings: Extract most of test_settings to test_extra_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:29:50 -07:00
Anders Kaseorg 5546762bd9 settings: Extract computed settings to computed_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:29:50 -07:00
Anders Kaseorg 826ca3bfdf settings: Move LDAP_DEACTIVATE_NON_MATCHING_USERS default to default_settings.
Tweaked by tabbott to fix an incorrect translation to ONLY_SSO.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:27:55 -07:00
Anders Kaseorg 24d320f245 dev_settings: Move prod_settings_template import to configured_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:20:42 -07:00
Anders Kaseorg 34b03dcf8a settings: Move DEBUG default to default_settings.
This is overridden in the prod_settings_template.py generated by
build-release-tarball.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:20:42 -07:00