Commit Graph

39466 Commits

Author SHA1 Message Date
Anders Kaseorg a109d68af0 create-test-api-docs: Remove unhelpful exception handler.
encode returns bytes, which cannot possibly be written to a text file.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:35 -07:00
Anders Kaseorg 234f7245cf export_usermessage_batch: Use os.rename.
This avoids an extra stat call to check whether the target is a
directory.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:35 -07:00
Anders Kaseorg 83eca256a4 compilemessages: Use polib for get_name_from_po_file.
This also corrects the name of zh_TW from “Chinese” to
“Chinese (Taiwan)”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:35 -07:00
Anders Kaseorg 1346c5397a zephyr: Use correct shell quoting for ssh.
ssh always runs its command through a shell (after naïvely joining
multiple arguments with spaces), so it needs an extra level of shell
quoting.  This should have no effect because we already validated user
with a regex, but it’s better for escaping to be locally correct in
case the context changes.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:35 -07:00
Anders Kaseorg 82593338ba report: Show Git commit in a way that works for merges.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:35 -07:00
Anders Kaseorg c9fec8f021 deliver_scheduled_messages: Don’t do_send_messages inside a transaction.
do_send_messages has side effects outside the database and may not
work reliably if its database effects are reordered by being inside a
transaction.

This also fixes a bug where we were doing the update incorrectly on
the Message table.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:09:22 -07:00
Alex Vandiver c2132a4f9c queue: Drop register_json_consumer / json_drain_queue interface.
Now that all callsites use the same interface, drop the now-unused
ones, and their tests.
2020-10-11 14:19:42 -07:00
Alex Vandiver 5477b9d9a1 queue: Switch tests to start_json_consumer interface. 2020-10-11 14:19:42 -07:00
Alex Vandiver 179c387409 tornado: Switch to start_json_consumer interface. 2020-10-11 14:19:42 -07:00
Alex Vandiver f0b23b0752 queue: Switch non-batch consumer to also use start_json_consumer.
This has no effect on consumption rate, but unifies the codepaths.
Before:
```
$ ./manage.py queue_rate --count 50000
Purging queue...
Enqueue rate: 11187 / sec
Dequeue rate: 4158 / sec
```

After:
```
$ ./manage.py queue_rate --count 50000
Purging queue...
Enqueue rate: 11010 / sec
Dequeue rate: 4113 / sec
```
2020-10-11 14:19:42 -07:00
Alex Vandiver 45c9c3cc30 queue: Monitor user_activity queue, now that it has a consumer.
Since this was using repead individual get() calls previously, it
could not be monitored for having a consumer.  Add it in, by marking
it of queue type "consumer" (the default), and adding Nagios lines for
it.

Also adjust missedmessage_emails to be monitored; it stopped using
LoopQueueProcessingWorker in 5cec566cb9, but was never added back
into the set of monitored consumers.
2020-10-11 14:19:42 -07:00
Alex Vandiver f9358d5330 queue: Switch batch interface to use the channel.consume iterator.
This low-level interface allows consuming from a queue with timeouts.
This can be used to either consume in batches (with an upper timeout),
or one-at-a-time.  This is notably more performant than calling
`.get()` repeatedly (what json_drain_queue does under the hood), which
is "*highly discouraged* as it is *very inefficient*"[1].

Before this change:
```
$ ./manage.py queue_rate --count 10000 --batch
Purging queue...
Enqueue rate: 11158 / sec
Dequeue rate: 3075 / sec
```

After:
```
$ ./manage.py queue_rate --count 10000 --batch
Purging queue...
Enqueue rate: 11511 / sec
Dequeue rate: 19938 / sec
```

[1] https://www.rabbitmq.com/consumers.html#fetching
2020-10-11 14:19:40 -07:00
Alex Vandiver 571f8b8664 queue: Use low-level queue_purge to empty at the end of tests.
This is O(1) at the RabbitMQ API level, and doesn't rely on the code
under test to function correctly during test cleanup.
2020-10-09 20:43:49 -07:00
Alex Vandiver ac0ba21c2c tests: Stop reusing a variable name.
`loopworker_sleep_mock` is a file-level variable used to mock out the
sleep() call in LoopQueueProcessingWorker; don't reuse the variable
name for something else.
2020-10-09 20:42:20 -07:00
Alex Vandiver 754638f673 tests: Refactor test_queue_worker to separate queues. 2020-10-09 20:42:12 -07:00
Alex Vandiver 2547bdbf4a queue: Rename consume_wrapper to a better name. 2020-10-09 20:40:51 -07:00
Alex Vandiver d5a6b0f99a queue: Rename queue_size, and update for all local queues.
Despite its name, the `queue_size` method does not return the number
of items in the queue; it returns the number of items that the local
consumer has delivered but unprocessed.  These are often, but not
always, the same.

RabbitMQ's queues maintain the queue of unacknowledged messages; when
a consumer connects, it sends to the consumer some number of messages
to handle, known as the "prefetch."  This is a performance
optimization, to ensure the consumer code does not need to wait for a
network round-trip before having new data to consume.

The default prefetch is 0, which means that RabbitMQ immediately dumps
all outstanding messages to the consumer, which slowly processes and
acknowledges them.  If a second consumer were to connect to the same
queue, they would receive no messages to process, as the first
consumer has already been allocated them.  If the first consumer
disconnects or crashes, all prior events sent to it are then made
available for other consumers on the queue.

The consumer does not know the total size of the queue -- merely how
many messages it has been handed.

No change is made to the prefetch here; however, future changes may
wish to limit the prefetch, either for memory-saving, or to allow
multiple consumers to work the same queue.

Rename the method to make clear that it only contains information
about the local queue in the consumer, not the full RabbitMQ queue.
Also include the waiting message count, which is used by the
`consume()` iterator for similar purpose to the pending events list.
2020-10-09 20:40:39 -07:00
Alex Vandiver a1ce1aca3b queue: Update comment to be more accurate about import errors. 2020-10-09 20:40:32 -07:00
Alex Vandiver 2d71ca1fb8 email: Remove unused `log_digest_event` function.
Its last callsite was removed in e46cbaffa2.

Also ref #6786.
2020-10-08 20:35:53 -07:00
Anders Kaseorg f0ae2a88f2 activity: Show server time rather than browser time.
This gets rid of an inline <script> that was blocking #16451.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-08 19:17:35 -07:00
Sumanth V Rao 264bf1fc2a css: Hide bottom-right resizable box in view-source textarea. 2020-10-08 17:13:50 -07:00
Sumanth V Rao adfe4cacc6 css: Add vertical scrollbar styling for view-source textarea.
This styles the vertical scrollbar similar to the horizontal
one for <pre/> (which can be seen in narrow windows). Strictly
speaking, this change shouldn't go in rendered_markdown.scss, but
placing it there helps unify the two scrollbar stylings rather than
duplicating them.
2020-10-08 17:13:50 -07:00
Sumanth V Rao ab9a011f89 css: Modify button styling to adjust for svg size change.
This builds on the previous commit and adds final touches to
both the button positioning and style.
2020-10-08 17:13:50 -07:00
Sumanth V Rao b0c37713e5 templates: Extract and tweak copy_to_clipboard_svg from both buttons.
Since both the original button svg's have their own individual height
and width, this commit tweaks it to values height=20 and width=16 which
works well for both buttons.
2020-10-08 17:13:50 -07:00
Sumanth V Rao 413df1881e templates: Extract and tweak copy_message_button from message_edit_form.
Small tweak in the extracted text which adds aria-lable to the button.
2020-10-08 17:13:50 -07:00
Sumanth V Rao 792029cbb8 css: Refactor copy_message button to use copy_button_base class styling.
Z-index is added to the base class. This doesn't affect copy_code_button
in any way.

Attributes dropped/changed:
    - background-color
    - Base class on-hover property is now used.
    - height, width, padding is now the base classes.

We can also remove the TODO now.
2020-10-08 17:13:50 -07:00
Sumanth V Rao 26ff6db751 css: Extract copy_button_base from copy_codeblock class.
The base class will contain common styling which is used by both
copy_codeblock and copy_message buttons. This sets us up nicely
for following commit(s) which aims to unify the two button styling.
2020-10-08 17:13:50 -07:00
sahil839 4e8f5b5b31 streams: Change access_stream_for_delete_or_update to also return sub.
We modify access_stream_for_delete_or_update function to return
Subscription object also along with stream. This change will be
helpful in avoiding an extra query to get subscription object in
code for updating subscription role.
2020-10-08 17:07:30 -07:00
Aman Agrawal 623fb63406 recent_topics: Go to topic if user clicks anywhere in topic col. 2020-10-08 17:05:06 -07:00
Aman Agrawal d3f4f64b0e timerender: Extract func to calculate days_old from time. 2020-10-08 17:05:06 -07:00
Aman Agrawal c7d66421f9 sidebars: Fill search boxes to full width.
After our bootstrap upgrade, search boxes were not full width on
when sidebars acted as popovers on small widths.
2020-10-08 17:03:56 -07:00
Aman Agrawal 8b419c93e4
message_send: Fix old guests being treated as full members.
For streams in which only full members are allowed to post,
we block guest users from posting there.

Guests users were blocked from posting to admin only streams
already. So now, guest users can only post to
STREAM_POST_POLICY_EVERYONE streams.

This is not a new feature but a bugfix which should have
happened when implementing full member stream policy / guest users.
2020-10-08 11:30:11 -07:00
Aman Agrawal 6fe82620df login: Fix alignment of error message in login form. 2020-10-07 16:18:54 -07:00
Anders Kaseorg f81a5e87ed run-dev: Wait for children to exit on Ctrl+C after killing them.
In addition to being generally more correct, this works around a bug
in Node.js that causes webpack-dev-server to corrupt the terminal
state when exiting as a background process.

https://github.com/nodejs/node/issues/35536

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:15:19 -07:00
Anders Kaseorg 1c7db528bd test_server: Wait for run-dev after terminating it.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:15:19 -07:00
Anders Kaseorg e1a3aa9c1a deliver_scheduled_messages: Exit cleanly on Ctrl+C.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:15:19 -07:00
Anders Kaseorg 363374346c management: Use signal.pause to sleep forever.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:15:19 -07:00
Anders Kaseorg 81d21068b5 eslint: Fix no-useless-concat.
https://eslint.org/docs/rules/no-useless-concat

And add some escaping to static/js/markdown.js while I’m here.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg 043c34d944 eslint: Fix no-throw-literal.
https://eslint.org/docs/rules/no-throw-literal

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg 5c8117a426 eslint: Fix no-implicit-coercion.
https://eslint.org/docs/rules/no-implicit-coercion

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg 4fe06a141e eslint: Enable eslint-plugin-unicorn.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg 0042cf51c1 eslint: Fix unicorn/better-regex.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/better-regex.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg a37616e8c6 eslint: Fix unicorn/import-style.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/import-style.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg 65f86274b2 eslint: Fix unicorn/prefer-dataset.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/prefer-dataset.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg e68595a3fc eslint: Fix unicorn/no-reduce.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/no-reduce.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg 24d4ab327c eslint: Fix unicorn/prefer-negative-index.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/prefer-negative-index.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg 62dfee4930 eslint: Fix unicorn/prefer-node-append and unicorn/prefer-node-remove.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/prefer-node-append.md
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/prefer-node-remove.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg 8098acb63c eslint: Fix unicorn/prefer-trim-start-end.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/prefer-trim-start-end.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg 78ccb15d9e eslint: Fix unicorn/no-zero-fractions.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/no-zero-fractions.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Anders Kaseorg 0d4af4f8e7 eslint: Fix unicorn/prefer-type-error.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/prefer-type-error.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00