Commit Graph

35101 Commits

Author SHA1 Message Date
Steve Howell d916cbbb70 cosmetic: Remove ugly bot_data__* names. 2020-03-24 20:40:19 -07:00
Tim Abbott d9bb6d0081 compatibility: Add more strict desktop app blocking.
This allows us to block use of the desktop app with insecure versions
(we simply fail to load the Zulip webapp at all, instead rendering an
error page).

For now we block only versions that are known to be both insecure and
not auto-updating, but we can easily adjust these parameters in the
future.
2020-03-24 20:33:11 -07:00
Pragati Agrawal 6fb438277e settings_notifications: Use single `.on()` instead of multiple `.click()`.
By this change we don't need to loop over each input element to attach the
handler, instead, we can use single `.on()` which takes care of changes.
2020-03-24 16:06:45 -07:00
Pragati Agrawal 0080499b08 settings_notifications: Use get_input_element_value for input values.
settings_org.get_input_element_value function returns values of input
elements in proper format based on their data-setting-widget-type.
2020-03-24 16:06:45 -07:00
Pragati Agrawal 53294928c4 settings: Fix toggling push notification checkbox when disabled.
We know that by default the value of `enable_stream_push_notifications` is
true, so on some server if push notifications are disabled then due to
```
{{#unless show_push_notifications_tooltip}}
```
the `enable_stream_push_notifications` checkbox is rendered as unchecked
and when we update any other setting in the "Stream notifications" section,
due to `settings_notifications.update_page()` all checkboxes are changed to
current values which means above checkbox get checked.

We can deal with this bug either making changes in
`settings_notifications.update_page()` but we should show check/uncheck
state of checkboxes anyway so removing this `unless` condition fixes this
bug.
2020-03-24 16:06:45 -07:00
Pragati Agrawal 83933034a4 settings: Simplify logic to show push notification tooltip in templates.
Instead of having logical expressions in templates, it's always preferred
to calculating them in javascript and pass the results as a context. It
also enhances the readability of templates and testing of such logic is
easier in js over templates.
2020-03-24 16:06:45 -07:00
Pragati Agrawal 1871443d10 settings_notifications: Rename variable to all_notification_settings.
`all_notification_settings_labels` is misleading that this variable is a
list of notifications setting labels so changed it to
`all_notification_settings`.
2020-03-24 16:06:45 -07:00
Pragati Agrawal b8945bafb4 settings_org: Extract get_input_element_value function.
The reason for extracting this function is that getting the text, integer,
boolean value from the input elements (like checkboxes, dropdowns) is a
common task, and later we can use this function to get the input element
value in `settings_notifications` in the upcoming commit.
2020-03-24 16:06:45 -07:00
Ryan Rehman f439bd7c36 list_render: Reset Data for sorted tables.
This is a bug fix where, if a list_render
object with the given name exists and it's items
have been sorted, then the filtered_list's data
does not get updated on re-rendering.

This line was present in the original commit
9576d5caef.
2020-03-24 16:06:45 -07:00
Ryan Rehman 1c605366ed list_render: Remove requirement of filter in opts.
The use case for this are small or fixed tables, which do not need
filtering support. Thus we are able to not include the unnecessary
search input inside the html parent container.
It is not used at present, but will be required when we refactor
the settings pages.

We also split out exports.validate_filter function for
unit testing the above condition.
2020-03-24 16:06:45 -07:00
arpit551 0c821424cd lint: Run whitespace linter against .yml files.
Lint now checks for .yml files with whitespace_rules.
2020-03-24 15:31:59 -07:00
arpit551 6c1f1e12fe yml: Fix trailing whitespace in .circleci/config.yml.
Removed trailing whitespace in .circleci/config.yml.
2020-03-24 15:31:59 -07:00
Tim Abbott 85c9ffd91c message: Validate propagate_mode parameters.
This improves the error handling for invalid values of the
propagate_mode parameter to our message editing endpoints.
Previously, invalid values would just work like change_one rather than
doing nothing.
2020-03-24 12:36:45 -07:00
Tim Abbott f658e14139 node tests: Fix broken settings_org test.
This was broken in 6ab1908214.
2020-03-24 11:54:09 -07:00
Tim Abbott 7702140521 node tests: Add stream test for filter_with_new_params.
Extracted from work by Wbert Adrián Castro Vera.
2020-03-24 11:50:59 -07:00
shubhamgupta2956 6ab1908214 settings: Reword setting name in settings menu.
Rewords "User list on left sidebar in narrow windows" in settings menu
to "Show user list on left sidebar in narrow windows".
2020-03-24 10:47:00 -07:00
Anders Kaseorg 39f9abeb3f python: Convert json.loads(f.read()) to json.load(f).
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-24 10:46:32 -07:00
Siddharth Varshney 328961a4fc settings: Reorganize options/sections in `Organization Permissions`.
As a consequence of too many options in the bottom `Other permissions`
subsection, the `Save` button could end up too far up from the bottom,
such that it might appear offscreen on low-height laptops.

We fix this by reorganizing the settings in a way that is both more
intuitive and also ensures that none of the subsections are too tall.

Fixes: #14274.
2020-03-24 10:36:40 -07:00
shubhamgupta2956 24603134f4 css: Use SCSS nesting for ".header-main".
Used SCSS nesting for ".header-main" in archive.scss.
2020-03-24 10:27:11 -07:00
Mateusz Mandera 5ae6f4f0dd tornado: Put port in logging_data before setup_event_queue in runtornado.
setup_event_queue() generates some logs about loaded event queues, and
it's good for the logging system to have access to the port at that
point already.
2020-03-24 10:25:01 -07:00
Mateusz Mandera eb0216c5a8 middleware: Log <user.id>@subdomain instead of subdomain/<user.id>.
It was decided that the new format is preferable.
2020-03-24 10:25:01 -07:00
Mateusz Mandera a1daf0cf83 middleware: Log 'root/<user.id>' when realm string_id is ''. 2020-03-24 10:25:01 -07:00
Dinesh fb361c7257 auth: Set `development_only=True` for `SOCIAL_AUTH_GITLAB_KEY`.
In development env, we use `get_secret` to get
`SOCIAL_AUTH_GITLAB_KEY` from `dev-secrets.conf`. But in
production env, we don't need this as we ask the user
to set that value in `prod_settings_template.py`.

This restricts the code from looking `zulip-secrets.conf`
for `social_auth_gitlab_key` in production env.
2020-03-23 16:03:12 -07:00
Steve Howell 3e199ed559 reactions: Remove name from local_id.
Adding the emoji name, which can have multiple aliases,
to the `local_id` key seemed to needlessly complicate
the code here.
2020-03-23 17:30:18 -04:00
Steve Howell f59cca2dcd reactions: Rewrite code to use clean reactions.
Before this commit, the reactions code would
take the `message.reactions` structure from
the server and try to "collapse" all the reactions
for the same users into the same reactions,
but with each reaction having a list of user_ids.
It was a strangely denormalized structure that
was awkward to work with, and it made it really
hard to reason about whether the data was in
the original structure that the server sent or
the modified structure.

Now we use a cleaner, normalized Map to keep
each reaction (i.e. one per emoji), and we
write that to `message.clean_reactions`.

The `clean_reactions` structure is now the
authoritatize source for all reaction-related
operations.  As soon as you try to do anything
with reactions, we build the `clean_reactions`
data on the fly from the server data.

In particular, when we process events, we just
directly manipulate the `clean_reactions` data,
which is much easier to work with, since it's
a Map and doesn't duplicate any data.

This rewrite should avoid some obscure bugs.

I use `r` as shorthand for the clean reaction
structures, so as not to confuse it with
data from the server's message.reactions.

It also avoids some confusion where we use
`reaction` as a var name for the reaction
elements.
2020-03-23 17:30:18 -04:00
Tim Abbott 3d0b720543 provision: Fix missing apt update for installing new dependencies.
Previously, we only did apt updates when our sources.list files or
keys changed, which could result in provisioning errors for
development systems that don't routinely update their apt cache
(probably including ~all Vagrant environments).
2020-03-23 13:47:17 -07:00
arpit551 dc6b09dc04 requirements: Add importlib-metadata and importlib-resources in dev.in.
importlib-metadata and importlib-resources are dependent packages for jsonschema
and cfn-lint respectively. They are built-in modules in later versions
of python (3.8, 3.7). When update-locked-requirements is run within python3.7 or
3.8 they will generate difference in locked files so we build these modules separately
to avoid such conflicts.
2020-03-23 10:28:21 -07:00
arpit551 70bbb4ca7f requirements: Upgrade transifex-client to the latest version.
transifex-client0.13.4 did not support python3.8 so updated
it to the latest version. Earlier we kept transifex-client version to
0.13.4 as transifex-client0.13.5 added overly strict version bounds
on six and urllib3. With the latest version this is not the case.
Bumped provision version.
2020-03-23 10:28:21 -07:00
arpit551 dc6c362e4a api_docs: Replaced cgi.escape with html.escape.
cgi.escape is deprecated in python3.2 and removed in python3.8.
This function was unsafe because quote is false by default, hence
removed and replaced with a safer html.escape.
2020-03-23 10:28:21 -07:00
arpit551 258de497a8 setup_venv: Used get_venv_dependencies() to return correct VENV dependencies.
Used get_venv_dependencies function to return the correct dependencies
for RHEL, Centos, Fedora rather than importing them as separate
COMMON_YUM_DEPENDENCIES in provision and create-production-venv.
2020-03-23 10:28:21 -07:00
arpit551 25e091a065 setup_path: Fix KeyError in setup_path for virtualenv ≥ 20.
In virtualenv ≥ 20, the site_packages variable was removed from
activate_this.py.  To avoid a KeyError, replace
activate_locals['site_packages'] with os.path.join(venv, 'lib',
python_version), where python_version is the 'pythonX.Y' name of the
directory where site-packages resides in the virtualenv.

Fixes #14025.
2020-03-23 04:06:19 -07:00
Tim Abbott 6df86dab3e jira: Handle comment_created events without issue details.
I'm not sure what causes some Jira webhook events to not include the
metadata that other events do, but it's definitely a format sent by
real installations of Jira (likely a very old version, since this has
fields missing from what modern Jira does) and we've seen it in
production.

The best we can do is encourage users to upgrade Jira for better data.
2020-03-22 21:43:21 -07:00
Tim Abbott 180d8abed6 messages: Fix unlikely exception when trying to delete a message. 2020-03-22 21:35:27 -07:00
Tim Abbott 481d351cee events: Fix buggy apply_events handling of starred_messages.
The previous starred_messages race handling did not correctly consider
the possibility that an event queue might have been registered without
starred_messages.
2020-03-22 21:30:23 -07:00
Mateusz Mandera ea93810d9a check-rabbitmq-queue: Put user check before rabbitmqctl call. 2020-03-22 18:46:28 -07:00
Mateusz Mandera 5da2f80140 queue_processors: Extract a duplicated logic block into do_consume. 2020-03-22 18:45:46 -07:00
Mateusz Mandera 27c19b081b rate_limit: Remove inaccurate docstring on clear_history methods. 2020-03-22 18:42:35 -07:00
Mateusz Mandera b9e5103d0c rate_limit: Refactor RateLimiterBackend to operate on keys and rules.
Instead of operating on RateLimitedObjects, and making the classes
depend on each too strongly. This also allows getting rid of get_keys()
function from RateLimitedObject, which was a redis rate limiter
implementation detail. RateLimitedObject should only define their own
key() function and the logic forming various necessary redis keys from
them should be in RedisRateLimiterBackend.
2020-03-22 18:42:35 -07:00
Mateusz Mandera 8069133f88 rate_limit: Remove __str__ methods of RateLimitedObjects.
These were clunky from the start and are no longer used, as keys are now
used directly for logging purposes.
2020-03-22 18:42:35 -07:00
Mateusz Mandera 4e9f77a6c4 rate_limit: Adjust keys() of some RateLimitedObjects.
type().__name__ is sufficient, and much readable than type(), so it's
better to use the former for keys.
We also make the classes consistent in forming the keys in the format
type(self).__name__:identifier and adjust logger.warning and statsd to
take advantage of that and simply log the key().
2020-03-22 18:42:35 -07:00
Mateusz Mandera 2c6b1fd575 rate_limit: Rename key_fragment() method to key(). 2020-03-22 18:42:35 -07:00
Mateusz Mandera 9c9f8100e7 rate_limit: Add the concept of RateLimiterBackend.
This will allow easily swapping and using various implementations of
rate-limiting, and separate the implementation logic from
RateLimitedObjects.
2020-03-22 18:42:35 -07:00
Mateusz Mandera 85df6201f6 rate_limit: Move functions called by external code to RateLimitedObject. 2020-03-22 18:42:35 -07:00
Mateusz Mandera 2b51b3c6c5 middleware: Also log request subdomain when logging "unauth" request.
This returns us to a consistent logging format regardless of whether
the request is authenticated.

We also update some log examples in docs to be consistent with the new
style.
2020-03-22 18:32:04 -07:00
Mateusz Mandera 3b5b19fde8 tornado: Log shard id in all logs coming from tornado processes.
This will make it easier to investigate using logs which requests are
being processed by which Tornado process.
2020-03-22 18:26:35 -07:00
Dinesh 5cb476e03d auth: Handle confirm registration page in `stage_two_of_registration`.
When a user in login flow using github auth chooses a email that is
not associated with an existing account, it leads to a "continue to
registration" choice. This cannot be tested with the earlier version
of `stage_two_of_registration`.
Also added the test.
Thanks to Mateusz Mandera for the solution.

Co-authored-by: Mateusz Mandera <mateusz.mandera@protonmail.com>
2020-03-22 17:31:01 -07:00
Dinesh 3de646d2cf auth: Improve GitHub auth with multiple verified emails.
The previous model for GitHub authentication was as follows:

* If the user has only one verified email address, we'll generally just log them in to that account
* If the user has multiple verified email addresses, we will always
  prompt them to pick which one to use, with the one registered as
  "primary" in GitHub listed at the top.

This change fixes the situation for users going through a "login" flow
(not registration) where exactly one of the emails has an account in
the Zulip oragnization -- they should just be logged in.

Fixes part of #12638.
2020-03-22 17:31:01 -07:00
Dinesh 5888d7c0f5 auth: Change how config error URLs are configured.
URLs for config errors were configured seperately for each error
which is better handled by having error name as argument in URL.
A new view `config_error_view` is added containing context for
each error that returns `config_error` page with the relevant
context.
Also fixed tests and some views in `auth.py` to be consistent with
changes.
2020-03-22 17:15:18 -07:00
Tim Abbott f644f5fc2c css: Change emoji size within text to 20x20px.
We've often gotten the complaint that Zulip's emoji are a bit too big;
this should address the worst consequences of that (line-wrapping
being off with large emoji present) while still making it possible to
easily see what a given emoji is.

The right place to change this is in rendered_markdown.scss, not the
main emoji definition in zulip.scss, as the latter is also used in
places like the emoji picker where a larger size is valuable.

Closes #12731, an older PR that did this with slightly different
parameters (and without a comment).
2020-03-22 15:09:49 -07:00
Siddharth Varshney 5a1b27bf32 css: Merge `.help .sidebar h2` in portico.scss. 2020-03-22 12:48:51 -07:00