Commit Graph

35265 Commits

Author SHA1 Message Date
YashRE42 556a13b7d6 portico: Drop font-weight for links from landing-page.scss.
This commit removes "font-weight: 500;" from landing-page.scss so as
to fix a bug on landing pages that used the `markdown` class to
format content. The bug was caused by "a:hover" from landing-page.scss
overriding the font-weight (600) on links as set by the markdown
class, this caused the text to seem jumpy when one hovered over links.

Note from tabbott: The original code was added in
d7f5f31f6a, which doesn't explain it's
purpose, but it predates the more complete "markdown" CSS, was part of
an early prototype that had unfortunate hover behavior more generally,
and makes sense to remove.

Fixes: #14387.
2020-03-30 12:55:23 -07:00
Steve Howell 2fe21de733 dev login: Sort users. 2020-03-30 10:16:27 -04:00
Steve Howell a37c183728 bot tests: Add test_multiple_services().
This test would have failed before we fixed the
bug in #14384.
2020-03-29 15:12:27 -07:00
Steve Howell 123604e4ba Improve test_pm_to_outgoing_webhook_bot().
We now have Hamlet, not Othello, send the message
to Othello's bot, since that's a more interesting
test and less likely to lead to a false positive.

And then we simplify the recipient check to avoid
the strange mypy mess as well as possible false
negatives.
2020-03-29 15:12:27 -07:00
Steve Howell fd77cfc4df tests: Inline TestOutgoingWebhookMessaging.setUp().
It's more readable if you just have the two
lines at the top of each function telling you
who's the bot owner and how you're creating the
bot.
2020-03-29 15:12:27 -07:00
Udit107710 ef741bf317 messages: Return shallow copy of message object.
When more than one outgoing webhook is configured,
the message which is send to the webhook bot passes
through finalize_payload function multiple times,
which mutated the message dict in a way that many keys
were lost from the dict obj.

This commit fixes that problem by having
`finalize_payload` return a shallow copy of the
incoming dict, instead of mutating it.  We still
mutate dicts inside of `post_process_dicts`, though,
for performance reasons.

This was slightly modified by @showell to fix the
`test_both_codepaths` test that was added concurrently
to this work.  (I used a slightly verbose style in the
tests to emphasize the transformation from `wide_dict`
to `narrow_dict`.)

I also removed a deepcopy call inside
`get_client_payload`, since we now no longer mutate
in `finalize_payload`.

Finally, I added some comments here and there.

For testing, I mostly protect against the root
cause of the bug happening again, by adding a line
to make sure that `sender_realm_id` does not get
wiped out from the "wide" dictionary.

A better test would exercise the actual code that
exposed the bug here by sending a message to a bot
with two or more services attached to it.  I will
do that in a future commit.

Fixes #14384
2020-03-29 15:12:27 -07:00
Steve Howell 4c51a94bcd message: Move transitional shim for delivery email.
If we have an old event that's missing the field
`sender_delivery_email`, we now patch it at the top
of `process_message_event`, rather than for each call
to `get_client_payload`.  This will make an upcoming
commit a bit easier to reason about.  Basically, it's
simpler to shim the incoming event one time rather
than doing it up to four times.  We know that
`get_client_payload` is non-destructive, because it
does a deepcopy.
2020-03-29 15:12:27 -07:00
Steve Howell 8002f48099 tests: Validate payload in test_build_bot_request.
We now validate the message data explicitly, rather
than comparing it to the event data.  This protects
us from false positives where we were only validating
that the request data was a mutated version of the
event message data.  (We'll have a commit soon that
fixes a mutation-related bug.)
2020-03-29 15:12:27 -07:00
Steve Howell 50c197c1fc tests: Move bot event code out of setUp().
This code is only used in one test, and having
the indirection of setUp partly obscured a
problem with the fact that our event message
is actually a wide dict that gets mutated
by `build_bot_request`.  We'll fix that soon,
but this is a pure code move for now.
2020-03-29 15:12:27 -07:00
Steve Howell f329fe465d minor: Use local instead of class `self.bot_user`. 2020-03-29 15:12:27 -07:00
Steve Howell e29ddd0ce0 outgoing_webhook: Remove `event` from process_success.
The `event` parameter is never used by `process_success`,
and eliminating it allows us to greatly simplify tests
that are just confusingly passing in events that are
totally ignored.
2020-03-29 15:12:27 -07:00
Steve Howell bacfadbc61 minor: Use explicit params in build_bot_request.
I also tweaked the block comment to mention
gravatars.
2020-03-29 15:12:27 -07:00
Steve Howell 6b86d9055f tests: Only wait up to 90s for the server. 2020-03-29 12:17:16 -07:00
Steve Howell cef74e1fdd tests: Sleep a bit more between server-is-up checks.
A watched pot never boils.

If we get server startup time down enough where 400ms
becomes a significant chunk, we can re-tweak this.
2020-03-29 12:17:16 -07:00
Steve Howell 7e4d542df4 tests: Use zulipdev.com for server_is_up() check.
Folks can have issues connecting to Casper
as zulipdev.com when they are not connected to
the internet or just have a bad connection, since
the DNS record is on the internet.  Folks can
work around this by just creating an /etc/hosts
entry for zulipdev.com, but people don't always
know.

This fix moves the symptom slightly earlier in
the process--we don't advertise that the server
is "up" if you can't actually connect to it as
"zulipdev.com".
2020-03-29 12:17:16 -07:00
orientor b9d0d6edcb openapi: Use response schema for describing simple success response.
In zulip.yaml simple json success response which only contains 'msg'
and 'result' properties has been described repeatedly in multiple
endpoints. Instead, use SimpleSuccess template for such responses
to increase code modularity and reusablility.
2020-03-29 19:25:14 +05:30
vaibhavrajsingh2001 51a8873579 widgets: Hide edit question icon in poll widget for non-author users.
If a non-author user clicked on view source in a poll and then close it,
the edit question icon would incorrectly get visible. This made changing
the question in local echo possible for non-author users.

Fixes: #14299
2020-03-28 15:04:36 -04:00
shubhamgupta2956 12474a3deb api docs: Migrate REAL-TIME /events to OpenAPI.
Migrate "call_on_each_event" from api/arguments.json to
/events:real-time in OpenAPI.

This is a bit of a hack, but it lets us eliminate this secondary
arguments.json file, which is probably worth it.

Tweaked by tabbott to fix various formatting issues in the original
documentation while I was looking at it.
2020-03-27 17:43:35 -07:00
shubhamgupta2956 a578234fdc api docs: Migrate /message/{message_id}:patch.
Most part of "/message/{message_id}" is migrated to OpenAPI. This commit
migrated the remaning payload "update-message-edit-permission-error"
from "api/fixtures.json" to OpenAPI. This commit also fixes an error
schema in "zulip.yaml" for this payload.
2020-03-27 17:29:29 -07:00
shubhamgupta2956 3096251995 api docs: Remove payload from fixtures.json.
Remove unused payload ("bad_event_queue_id_error",
"successful-response-empty") from api/fixtures.json.
2020-03-27 17:29:29 -07:00
Stefan Weil d2fa058cc1
text: Fix some typos (most of them found and fixed by codespell).
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2020-03-27 17:25:56 -07:00
orientor 6eb39c8668 openapi: Migrate remaining properties from zulip-2.0.yaml to zulip.yaml.
This moves all useful work from the temporary/testing file
zulip-2.0.yaml to zulip.yaml, allowing us to remove the old file.
2020-03-27 17:09:10 -07:00
arpit551 8f7733cb20 emails: Added placeholders strings in FormAddress.
We've had a bug for a while that if any ScheduledEmail objects get
created with the wrong email sender address, even after the sysadmin
corrects the problem, they'll still get errors because of the objects
stored with the wrong format.

We solve this by using FromAddress placeholders strings in
send_future_email function, so that ScheduledEmail objects end up
setting the final `from_address` value when mail is actually sent
using the setting in effect at that time.

Fixes #11008.
2020-03-27 16:41:02 -07:00
Vishnu KS 96eb1bcd9d tests: Test the case where Customer exists but stripe_customer_id is None. 2020-03-27 16:31:04 -07:00
Vishnu KS 388384e894 tests: Check for object equality instead of type equality.
I don't see any strong reason to do only type equality
when we can do object equality here.
2020-03-27 16:31:04 -07:00
Kartik Srivastava 1388a1d2b1
api docs: Refactor get_single_user to use get_user_by_id.
This refactors get_single_user to use get_user_by_id instead of
call_endpoint.  Doing so is only possible now that we've upgraded
python-zulip-api to a version with the new function.
2020-03-27 15:49:20 -07:00
Steve Howell f62f8c9238 message tests: Clean up edit-message tests.
Overall, this change eliminates a lot of
optional parameters and conditionals, plus
some legacy logic related to caches.

For all the places we are just editing topics,
we now just call `check_topic` to see that
the topic got updated.

For places where the topic edit failed, we
just inline the checks that message still
has the old topic and content.

And then for successful **content** edits,
we now do a more rigorous, more sane check
that the messages are properly cached.  The
old code here had evolved from 2013 into
something that didn't really make much sense
in the context of editing topics.

Now we are literally pulling data from the
cache and making sure it's valid, rather
than trying to poorly simulate the two
codepaths related to dispatching message
events and fetching messages.  Some of the
history here was that when I introduced
`MessageDict` several years ago, I did a
lot of code sweeping and didn't analyze every
single test to make sure it's still valid,
plus some of the tests still had some value
for catching regressions.  A recent commit
now gets us coverage on that a lot more
explicitly, rather than in passing.
2020-03-27 13:51:50 -07:00
Steve Howell db4ae7fc39 tests: Test equivalency of the two message codepaths.
See the comment in the test for a thorough explanation.

In brief, this test makes sure that the events codepath
for messages produces the same results as the fetch
codepath.

And this sets us up to simplify another test that kind
of poorly tried to do the same thing in passing.  (In
fairness the test was really ancient and preceded a lot
of later work that we did here.)
2020-03-27 17:20:34 +00:00
Steve Howell c2b3269420 message perf: Streamline stream name lookups.
When we are fetching messages, we need to hydrate
stream names into the messages for legacy reasons.

(Ideally, we could skip this step for the webapp
and modern mobile clients, since they really only
need stream_ids, but we're not there yet.)

We keep a recipient cache that maps recipient ids
to stream names.

When we populate that cache, we now use `values(...)`
to avoid fat objects and extra DB work.

Note that we are already using a similar technique
for hydrating PM/huddle recipients.
2020-03-27 17:20:34 +00:00
Tim Abbott 0f1bdcc46f restart-server: Restart Tornado processes individually.
After some testing, I've confirmed that this seems to behave
significantly better in terms of the number of failed requests due to
Tornado being the process of restarting compared with the previous
version, as each individual process is only down for a short time,
rather than all of them being down at once.
2020-03-27 06:23:34 -07:00
Tim Abbott cb1fb94ac8 webhooks: Fix Jira webhook constructing subject still too early.
For event types that we don't yet support, like worklog_created (and
likely many more in the future), it doesn't make sense to call a
function that only parses issue events correctly.
2020-03-27 06:19:00 -07:00
Tim Abbott 3abb52476c confirmation: Add database indexes on confirmation fields.
Apparently, we didn't have any database indexes on Confirmation, which
meant that on servers with large numbers of users like zulipchat.com,
new account registration could spend a ton of time effectively doing a
table scan on this table.
2020-03-27 02:03:05 -07:00
Tim Abbott 737a36a9f8 portico: Use a backend variable to determine desktop version.
This makes it possible to change this parameter without rebuilding all
the server's static assets.
2020-03-27 01:37:56 -07:00
thoemie 7a48bec189 docs: Document EMAIL_USE_SSL for smtp providers on port 465.
I found the solution by simply trying out EMAIL_USE_SSL and it
worked. I had problems with sending emails (did not work at all, there
wasn't even a connection going on - I checked with tcpdump. Then I
found this: To use port 465, you need to call
smtplib.SMTP_SSL(). Currently, it looks like Django only uses
smtplib.SMTP() (source: https://code.djangoproject.com/ticket/9575).

Fixes #14350.
2020-03-27 00:25:49 -07:00
Ryan Rehman b4ade7b6d8 migrations: Refactor the enum type fields.
Accessing attributes is the preferred design
pattern, as it is more readable.
2020-03-27 00:21:21 -07:00
sahil839 a2efe3ab64 narrow: Show starred messages from muted topics in starred message view.
Starred messages from muted topics were not shown in the starred
messages view. Condition for muting_enabled is modified accordingly
such that the starred messages from muted topics is shown in the
starred messages narrowed view.

Node tests are updated accordingly.

Fixes #13548
2020-03-27 00:17:26 -07:00
Tim Abbott 06c97b5be2 api docs: Render example responses as with JSON codehilite.
This makes the example responses a lot prettier visually.
2020-03-27 00:03:36 -07:00
Tim Abbott 820f0e275e api docs: Redesign visuals for documenting arguments.
The previous system for documenting arguments was very ugly if any of
the examples or descriptions were wrong.  After thinking about this
for a while, I concluded the core problem was that a table was the
wrong design element to use for API parameters, and we'd be much
better off with individual card-type widgets instead.

This rewrites the API arguments documentation implementation to use a
basic sort of card-like system with some basic styling; I think the
result is a lot more readable, and it's a lot more clear how we would
add additional OpenAPI details (like parameter types) to the
documentation.
2020-03-27 00:03:36 -07:00
Anders Kaseorg 43996106d8 requirements: Get transifex-client from apt.
In the past it has blocked Python library security updates with overly
strict version bounds, and we don’t use it as a library, only as a
binary.

Skip the PROVISION_VERSION bump because we can use the tx binary from
either location.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 21:39:04 -07:00
Anders Kaseorg 006a69c912 ci: Disable test-run-dev.
This test doesn’t catch many bugs and is known to be flaky.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 21:38:19 -07:00
Anders Kaseorg a6624f04db Revert "run-dev: Run process_queue with DJANGO_AUTORELOAD_ENV."
This reverts commit 36a8e61e67 (#13934).

The Django 2.2 autoreloader works by forking into a child process that
exits with status 3 when a file changes, and a parent process that
restarts the child when it exits with status 3.  Setting this
environment variable had the effect of pretending we were already the
child process, without a parent process to restart it.  Therefore,
changing any code used by the queue processor caused it to exit rather
than restart.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 21:38:19 -07:00
Anders Kaseorg 7ff9b22500 docs: Convert many http URLs to https.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 21:35:32 -07:00
Anders Kaseorg 586f78cb32 install-node: Upgrade to Node.js 12.16.1, Yarn 1.22.4, nvm 0.35.3.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 21:34:47 -07:00
Anders Kaseorg 56aadf6503 tests: Fix CI apocalypse perpetrated by previous commit.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 21:25:25 -07:00
Tim Abbott 9b9efdfa0f docs: Update API documention to note changes in Zulip 2.1.0. 2020-03-26 20:06:39 -07:00
Tim Abbott b29213ef90 docs: Document changes in the API made since Zulip 2.1.x.
This is important documentation for anyone working on writing clients
for the Zulip APIs that wants to support older releases.
2020-03-26 17:45:03 -07:00
Tim Abbott 9b2d4561e6 api docs: Document that the `GET /users` endpoint is new in 2.2.
I verified there are no other endpoints we've added since Zulip 2.1,
so at least this part of our documentation is accurate.

We added the `/realm/export` endpoints in Zulip 2.1, but those don't
have documentation yet, so we'll just need to remember to cover this
when they are added.
2020-03-26 17:22:30 -07:00
Eeshan Garg 1d5d0e649b pypi: Upgrade Zulip's PyPI packages to version 0.6.4. 2020-03-26 17:17:33 -07:00
Anders Kaseorg 58e9526586 tsconfig: Disable automatic inclusion of type packages.
Automatic inclusion is for files with global declarations, as opposed
to files declared as modules.  typeRoots already had the side effect
of disabling the default automatic inclusion for node_modules.  Since
we also don’t need it for static/js/types, we may as well disable it
completely.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 17:07:50 -07:00
Mateusz Mandera 1dd5392874 requirements: Bump python-social-auth to 3.3.2. 2020-03-27 00:00:24 +00:00