Commit Graph

34747 Commits

Author SHA1 Message Date
Pragati Agrawal c68d90fea1 settings user groups: Fix organization admin can not create user groups.
The bug was in complex `if` condition, which should mean that users should
be allowed to create a User group only when they are either admin or user
group creation policy is set to everyone.

Fixes: #13909.
2020-02-24 12:12:52 -08:00
Steve Howell 8abff20ce8 settings: Move time limit properties.
We have to extract a local `config` due to line
length restrictions.  (Or do something else ugly.)
2020-02-24 11:59:05 -08:00
Steve Howell 720e9728db settings_config: Move private_message_policy_values. 2020-02-24 11:53:33 -08:00
Steve Howell 3dd9638f01 settings_config: Move user_group_edit_policy_values. 2020-02-24 11:53:33 -08:00
Steve Howell b4304721a8 settings_config: Move *_stream_policy_values. 2020-02-24 11:53:33 -08:00
Steve Howell 591d8b6105 settings_config: Move email_address_visibility_values. 2020-02-24 11:53:33 -08:00
Tim Abbott af188205cb echo: Add a block comment explaining found_newest=False case.
This is just clarifing some details for postering from the previous
commit.
2020-02-24 11:50:50 -08:00
Steve Howell 6dba3f2420 local echo: Avoid echo until "newest" are found.
We now no longer do local echo if a user has logged in or visited a
narrow so recently that we are still fetching new messages for them in
their current message list.

Since we want any message list we're displaying to show only
contiguous sequences of messages within that view, it's not correct to
append messages that were just sent at the end unless
fetch_status.has_found_newest shows that we are up to date with the
latest messages from the server.

While we have some logic aimed at correcting our-of-order message IDs
in Zulip, even a brief (few seconds) temporary display of that is a
bug that we should avoid.

This means that we should disable local echo when the user's current
narrow is not up to date.  We can be sure that we'll get the message
the user sent from the server either during the catch-up process or
when we receive it back from th server via the events system.

That particular race window can be several seconds in situations where
somebody is in a narrow where their pointer (or equivalent) is far
behind the latest messages.

This commit only fixes the local echo race condition.  There's a
related bug where new messages sent by (potentially other) users
delivered to the client via server_events might race with our fetching
until we get the latest messages in a given narrow, which we'll need
to deal with separately.

See https://github.com/zulip/zulip/issues/8989 for more details.  It's
possible that we'll close the issue after this fix, since any
additional fixes would add a lot of complexity, and I'm not sure how
much of a problem this will really be in practice after this fix.

Note that we don't have great automated testing for
`try_deliver_locally` (or really `echo.js` in general).  For
`try_deliver_locally` the node tests would probably be 8x more complex
than the code itself, since that function is basically "glue" code
touching several external dependencies.  It's also kind of hard to
screw up this code without getting pretty obvious failures early in
the QA process.

Fixes #8989.
2020-02-24 11:45:00 -08:00
harshavardhanpb cac4feb263 openapi: Move openapi.py into zerver/openapi.py.
Fixes #14006
2020-02-24 12:21:26 -05:00
Steve Howell ed859617e4 minor: Add test for extract_stream_indicator. 2020-02-24 07:40:31 -05:00
Steve Howell ea14dff7de tictactoe: Fix type confusion.
With the new Map, we want to make sure we
convert the square number into an int.

The symptom here was you'd click on the
square, and the data would get passed
around via the event system, but when
we went to draw the board, the idx value
was a string.
2020-02-21 20:01:21 -05:00
Steve Howell 5e8279c2fb refactor: Extract settings_config.
This moves some code from settings_display.js
into the new module settings_config.js.

Extracting this module breaks some dependencies
on settings_display.js (which has some annoying
transitive dependencies, including jQuery).

In particular this isolates stream_data from
from settings_display.js.

Two of the three structures that we moved here
weren't even directly used by settings_display.js,
since we do a lot of rendering in the modules
admin.js and setting.js.

We make get_all_display_settings() a function
to avoid a require-time dependency on page_params.

Breaking the dependencies simplifies a few
node tests.

Most of the node test complexity came from the
following commit in March 2019:

5a130097bf

The commit itself seems harmless enough, but
dependencies can have a somewhat "viral" nature,
where making stream_data depend on settings_display
caused us to modify four different node tests.
2020-02-21 12:06:31 -08:00
Tim Abbott 8aae02de68 emoji: Add a block comment explaining spritesheet CSS. 2020-02-21 10:26:57 -08:00
Steve Howell af7923c557 refactor: Extract get_square_size() for emoji sprites.
This cleans up a few things:

    - just yield values so we don't have to do
      tedious max logic
    - use values() instead of items() for
      skin_variations loop

In the ideal world the emoji.json would reduce this
code to `get_square_size = lambda data: data['square_size']`,
but I don't think we can get the square size explicitly.
2020-02-21 10:21:12 -08:00
Steve Howell da1ce9a577 emoji sprites: Avoid hard coding CSS percentages.
This commit changes the calculation of the
background-size parameter that we use to
render emojis from sprite sheets.

In particular, it now makes the parameter
match the sizes of our latest sprite
sheets from Twitter/Google.

This should fix the geometry aspect of #13959,
but we also need to fix some issues with the
cache being sticky.

There is also some minor cleanup:

    - Remove obsolete -moz/-webkit CSS.
    - Remove needless precision in percentages.
    - Fix the transposed nrows/ncols names.
    - Add extensive commenting.

Finally, we add a minor bump to the provision
number.  This commit should be merged in the
same series as the other fix for this issue,
which will probably have a major bump, and we'll
need to rebase this appropriately.
2020-02-21 10:21:12 -08:00
majordwarf 621716bf30 portico: Remove footer-overlapping gradient from apps page.
This fixes an issue where the /apps page would have gradient colors
awkwardly overlapping the footer in mobile views.

This was because the /apps page was sharing /hello page gradient HTML
(defined in zerver/gradients.html), and the /apps content isn't tall
enough for the gradient content to be under actual content.

The fix is simple: Just don't include the gradient for /apps.  The
design for the page was long ago changed to not use the gradient.

We don't expect a similar bug with the gradients in other pages
because they all have enough content to have the gradients end well
before reaching the footer.

Fixes #13375.
2020-02-21 10:04:50 -08:00
Mateusz Mandera ac041956d9 test_cache: Test caching of None values is handled correctly. 2020-02-21 09:05:46 -08:00
Mateusz Mandera a9794ec001 cache: Delete unused function cache(). 2020-02-21 09:05:46 -08:00
Mateusz Mandera 9c99962dea saml: Add SOCIAL_AUTH_SAML_SECURITY_CONFIG to default_settings.
SOCIAL_AUTH_SAML_SECURITY_CONFIG["authnRequestsSigned"] override in
settings.py in a previous commit wouldn't work on servers old enough to
not have the SAML settings in their settings.py - due to
SOCIAL_AUTH_SAML_SECURITY_CONFIG being undefined.
This commit fixes that.
2020-02-21 10:19:25 -05:00
shubhamgupta2956 a05f633fc1 util: Replace util.set_message_topic().
Replace `util.set_message_topic(message, topic)` with `message.topic =
topic`.

Fixes #13931
2020-02-21 09:53:45 -05:00
shubhamgupta2956 efda2684ea util: Replace util.get_message_topic().
Replace `util.get_message_topic(message)` with `message.topic`.

Fixes #13931
2020-02-21 09:53:45 -05:00
Anders Kaseorg 9d598d95a6 puppet: Fix puppet-lint warning.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-20 19:51:48 -08:00
Mateusz Mandera bf0f1274fa saml: Make the bad idp param KeyError log message more verbose.
Original idea was that KeyError was only going to happen there in case
of user passing bad input params to the endpoint, so logging a generic
message seemed sufficient. But this can also happen in case of
misconfiguration, so it's worth logging more info as it may help in
debugging the configuration.
2020-02-20 14:49:27 -08:00
Mateusz Mandera 2faa2079f1 saml: Use rsa-sha256 as the default signature algorithm.
python3-saml uses the insecure rsa-sha1 as default.
2020-02-20 14:47:38 -08:00
Mateusz Mandera 83722275f2 docs: Rename "Testing and writing tests" section to "Testing overview".
The section doesn't really explain anything about actually writing
tests, so "Testing overview" seems like a more fitting name.
2020-02-20 17:21:20 -05:00
vsvipul 020a263a67 auth: Create a new page hop for desktop auth.
Create a new page for desktop auth flow, in which
users can select one from going to the app or
continue the flow in the browser.

Co-authored-by: Mateusz Mandera <mateusz.mandera@protonmail.com>
2020-02-20 11:59:55 -08:00
Tim Abbott aaee506fb2 provision: Always build pygments_data.json if missing.
While it's a bit of extra complexity to do this check, which I'm not
excited about, we've had multiple folks spend significant time being
confused rebasing past d7d8632525 into
deleting `pygments_data.json`, with provision not rebuilding it, so
this seems worth merging as a transitional fix even if we decide to
remove it in 2 months.
2020-02-20 10:25:11 -08:00
Mateusz Mandera c78d0712f7 tests: For ldap tests, give each ldap user a unique password.
To avoid some hidden bugs in tests caused by every ldap user having the
same password, we give each user a different password, generated based
on their uids (to avoid some ugly hard-coding in a bunch of places).
2020-02-19 14:46:29 -08:00
Vishnu KS 51f5701879 export: Canonicalize the email of cross realm bot to default value.
Fixes #13496
2020-02-19 14:44:50 -08:00
Anders Kaseorg fd2d642f95 postgres-init-db: Fix invocation from a relative path.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-19 14:40:37 -08:00
Anders Kaseorg cd43a33783 dependencies: Add source-map to top level devDependencies.
No PROVISION_VERSION bump is needed because it was already installed
indirectly.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-19 14:39:26 -08:00
Anders Kaseorg b019d7ffe8 typeahead: Convert to ES6 module.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-19 14:36:42 -08:00
Mateusz Mandera 7293862eae tests: Keep temporary .coverage files in var/ directory. 2020-02-19 14:36:17 -08:00
Mateusz Mandera 782353f0e1 tests: Ensure coverage data files get cleaned up at the end.
Without calling cov.erase() the data file seems to persist and even
pollute future test runs if not removed. Registering an atexit handler
seems like a good, and reasonably clean way to ensure the cleanup
happens.
Fixes #13933.
2020-02-19 14:36:17 -08:00
Anders Kaseorg 91edb7dc43 puppet: Fix regeneration of memcached-sasldb2 on password changes.
Puppet doesn’t re-run an exec blocks that’s declared as creating an
existing file, even if it’s notified.  Remove the creates declaration.

Fixes #13730.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-19 14:20:43 -08:00
Anders Kaseorg d918626135 restore-backup: Run generate_secrets.py.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-19 14:20:43 -08:00
Vishnu KS 0d4bf86130 management: Make backup command work when DB is not in localhost.
This is useful preparatory work for supporting the backup management
command inside docker-zulip.
2020-02-19 12:41:05 -08:00
Mateusz Mandera 7814f52d45 docs: Replace links to Django 1.11 docs with 2.2 links. 2020-02-19 11:51:18 -08:00
Tim Abbott b1608a51db docs: Update changelog with changes since last release.
This includes an experiment of having a draft of the 2.1.3 changelog,
which is helpful in avoiding duplication with the 2.2.0 changelog for
items we're planning to backport.
2020-02-19 11:50:47 -08:00
Vishnu KS e1a7716578 emails: Translate from_name of account security emails. 2020-02-18 17:45:33 -08:00
Anders Kaseorg 7990ef2d52 register: Spell Greek Ακμή with Greek alpha, not Latin A
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-18 17:41:43 -08:00
Tim Abbott 0075c6cd56 do_update_message: Clean up timestamp code.
By moving this logic to the topic of the functon, we make the code a
lot more readable.
2020-02-18 16:38:34 -08:00
Steve Howell b55d2bc256 markdown: Add helper configuration for mobile.
This refactoring is the first step toward sharing
our markdown code with mobile.  This focuses on
the Zulip layer, not the underlying third party `marked`
library.

In this commit we do a one-time initialization to
wire up the markdown functions, but after further
discussions with Greg, it might make more sense
to just pass in helpers on every use of markdown
(which is generally only once per sent message).
I'll address that in follow-up commits.

Even though it looks like a pretty invasive change,
you will note that we barely needed to modify the
node tests to make this pass.  And we have pretty
decent test coverage here.

All of the places where we used to depend on
other Zulip modules now use helper functions that
any client (e.g. mobile) can configure themselves.
Or course, in the webapp, we configure these from
modules like people/stream_data/hash_util/etc.

Even in places where markdown used to deal directly with
data structures from other modules, we now use functions.
We may revisit this in a future commit, and we might
just pass data directly for certain things.

I decided to keep the helpers data structure completely flat,
so we don't have ugly nested names like
`helpers.emoji.get_emoji_codepoint`.  Because of this,
some of the names aren't 1:1, which I think is fine.

For example, we map `user_groups.is_member_of` to
`is_member_of_user_group`.

It's likely that mobile already has different names
for their versions of these functions, so trying for
fake consistency would only help the webapp.  In some
cases, I think the webapp functions have names that
could be improved, but we can clean that up in future
commits, and since the names aren't coupled to markdown
itself (i.e. only the config), we will be less
constrained.

It's worth noting that `marked` has an `options`
data structure that it uses for configuration, but
I didn't piggyback onto it, since the `marked`
options are more at the lexing/parsing layer vs.
the app-data layer stuff that our helpers mostly
help with.

Hopefully it's obvious why I just put helpers in
the top-level namespace for the module rather than
passing it around through multiple layers of the
parser.

There were a couple places in markdown where we
were doing awkward `hasOwnProperty` checks for
emoji-related stuff.  Now we use the Python
principle of ask-forgiveness-not-permission and
just handle the getters returning falsy data.  (It
should be `undefined`, but any falsy value is
unworkable in the places I changed, so I use
the simpler, less brittle form.)

We also break our direct dependency on
`emoji_codes.json` (with some help from the
prior commit).

In one place I rename streamName to stream_name,
fixing up an ancient naming violation that goes
way back to before this code was even extracted
away from echo.js.  I didn't bother to split this
out into a separate commit, since 2 of the 4
lines would be immediately re-modified in the
subsequent commit.

Note that we still depend on `fenced_code`
via the global namespace, instead of simply
requiring it directly or injecting it.  The
reason I'm postponing any action there is that
we'll have to change things once we move
markdown into a shared library.  (The most
likely outcome is that we'll rename/move both files
at the same time and fix the namespace/require
details as part of that commit.)

Also the markdown code still relies on `_` being
available in the global namespace.  We aren't
quite ready to share code with mobile yet, but the
underscore dependency should not be problematic,
since mobile already uses underscore to use the
webapp's shared typing_status module.
2020-02-18 16:13:38 -08:00
Steve Howell e8de4abb0e markdown: Clean up userMentionHandler().
This mostly moves logic into people.js.
The people functions added here are glorified
two-liners.

One thing that changes here is that we
are a bit more rigorous about duplicate
names.

The code is slightly awkward, because this
commit preserves the strange behavior
that if 'alice|42' doesn't match on
the user with the name "alice" and user_id
"42", we instead look for a user whose
name is "alice|42".  That seems like a
misfeature to me, but there's a test for
it, so I want to check with Tim that it's not
intentional behavior before I simplify
the code.
2020-02-18 16:04:12 -08:00
Steve Howell be45809253 markdown: Extract emoji helpers.
We add this API to emoji.js, so that markdown
doesn't need to look at internal data structures
(or even need to understand any kind of record
format for results).

Here are the functions:

    get_realm_emoji_url()
    get_emoji_name()
    get_emoji_codepoint()

We use the API now in markdown, which eliminates
the need for the markdown parser to require
the emoji JSON file.

Each function has a simple docstring:

    get_emoji_name('1f384') === 'holiday_tree'
    get_emoji_codepoint('avocado') === '1f951'
    get_realm_emoji_url('shrug') === '/user_avatars/2/emoji/images/31.png'

Also we have simple test coverage for the API
(including tests that verify the docstrings).
2020-02-18 16:04:04 -08:00
Steve Howell f603710cd1 markdown: Pass in page_params.realm_filters to initialize(). 2020-02-18 15:52:34 -08:00
Steve Howell d55510b57d refactor: Rename set_realm_filters().
This name was misleading, because we weren't
actually setting realm_filters (that's what
`page_params.realm_filters = realm_filters`
is for); we were instead updating our
realm filter rules.
2020-02-18 15:52:34 -08:00
Steve Howell 43c1c285f2 node tests: Add explicit test for wrong-case mentions.
The current code lets you enter normal mentions with
the wrong casing, but if you do a duplicate-name mention
with ids, it does enforce case.
2020-02-18 15:52:34 -08:00
Pragati Agrawal 479d07c264 org settings: Convert for…of loop to .some.
This is a minor refactor to make the code cleaner, by replacing manual
looping with the Array.prototype.some() function.
2020-02-18 14:28:19 -08:00
Pragati Agrawal bbc25ab88b org settings: Fix usage of forEach method on jQuery object.
Commit 612b237cec introduced a
regression that broke the “Discard” button, because
get_subsection_property_elements returns a jQuery object rather than
array, and jQuery objects don’t have a forEach method.  Change it to
return an array.

[anders@zulipchat.com: Use Array.from instead of .toArray to avoid the
need for extra mocking.]
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-18 13:40:54 -08:00