Commit Graph

27241 Commits

Author SHA1 Message Date
Tim Abbott 47df8c0c6e notifications: Elimininate now-unnecessary database query.
Because we're passing through the trigger for notifications to
do_send_missedmessage_events_reply_in_zulip, we don't need to go back
to the database to determine which messages actually mentioned the
user.
2018-07-14 12:19:33 +05:30
Tim Abbott dabf3667d1 notifications: Eliminate unnecessary at_mention context variable.
we already had a variable "mention" that did the exact same thing.
2018-07-14 12:19:33 +05:30
Tim Abbott bc861b4441 notifications: Pass trigger to do_send_missedmessage_events_reply_in_zulip.
This change converts our logic for determining whether the current
user was mentioned in a group of messages from the implicit "if it was
sent to a stream, it's a mention" to the explicit "we actually know
there was a mention in the message".  This is an important
prerequisite for our upcoming feature to support getting email
notifications for streams always (even without a mention).
2018-07-14 12:02:08 +05:30
Tim Abbott 58a7a390c8 event_queue: Call build_offline_notification unconditionally.
Previously, maybe_enqueue_notifications had this very subtle logic,
where it set the notice variable only inside the block for push
notifications, but then also used it inside the block for email
notifications.

This "worked", because previously the conditions for push
notifications were always true if the conditions for email
notifications were, but the code was unnecessarily confusing.  The
only good reason to write it this way is if build_offline_notification
was expensive; in fact, the most expensive thing it does is calling
time.time(), so that reason does not apply here.

This was further confusing, in that in the original logic, we relied
on the fact that push notification code path edited the "notice"
dictionary for further processing.

Instead, we just call it separately and setup the data separately in
each code path.
2018-07-14 12:01:59 +05:30
Tim Abbott a09ebf0551 event_queue: Remove confusing comment about rabbitmq.
Whatever RabbitMQ check this comment used to be next to, it isn't next
to anymore.
2018-07-14 12:00:37 +05:30
Tim Abbott 08ed1b4605 notifications: Set trigger on offline email notifications.
This data will be required for correctly implementing the upcoming
stream_push_notify feature; it also helps support cleaning up the code
for the existing stream mentions logic.
2018-07-14 11:57:28 +05:30
Tim Abbott b0c393cf97 notifications: Format missed_messages argument as a dict.
Because in upcoming commits, we'll want to pass additional per-message
data into do_send_missedmessage_events_reply_in_zulip, we need to
expand the format for how we represent messages to account for that.
2018-07-14 11:34:04 +05:30
Aditya Bansal 2e837b1407 archives: Change the topic/<topic_name> code path to use topics prefix.
We do this for maintaining consistency in naming of the endpoints.
2018-07-14 10:39:35 +05:30
Tim Abbott bf1ad714da actions.py: Refactor generate_topic_history_from_db_rows for clarity.
This refactors the generate_topic_history_from_db_rows function to not
depend upon the assumption of rows passed as parameter to be sorted in
reverse order of max_message_id field.

Additionally, we add sorting and some tests that verify correct
handling of these cases.
2018-07-14 10:11:08 +05:30
Aditya Bansal 63ec8b08b8 archives: Add endpoint to fetch topic history of web public streams.
In this commit we add a new endpoint so as to have a way of fetching
topic history for a given stream id without having to be logged in.
This can only happen if the said stream is web public otherwise we
just return an empty topics list. This endpoint is quite analogous
to get_topics_backend which is used by our main web app.

In this commit we also do a bit of duplication regarding the query
responsible for fetching all the topics from DB. Basically this
query is exactly the same as what we have in the
get_topic_history_for_stream function in actions.py. Basically
duplicating now is the right thing to do because this query is
really gonna change when we add another criteria for filtering
messages which is:
Only topics for messages which were sent during the period the
corresponding stream was web public should be returned.
Now when we will do this, the query will change and thus it won't
really be a code duplication!
2018-07-14 09:51:37 +05:30
Eeshan Garg 37e7cb4b96 outgoing-webhooks.md: Use HTML tables to document payload fields. 2018-07-14 09:44:54 +05:30
Eeshan Garg 2e78a56796 outgoing-webhooks.md: Add example payloads/fixtures. 2018-07-14 09:44:54 +05:30
Eeshan Garg 67045d65a9 webhooks/gitlab: Include title in Issue Hook event messages.
We already include the issue title in the topic. But if one chooses
to group all gitlab notifications under one topic, the message body
is misleading in the sense that only the Issue ID and the description
are displayed, not the title, which isn't super helpful if the topic
doesn't tell you the title either.

I think we should err on the side of always including the title in
the main message body, which is what this commit does.

Fixes #9913.
2018-07-14 09:38:11 +05:30
Tim Abbott 811ac718d6 build_emoji: Document a quirk of generate_sprite_css_files. 2018-07-13 21:19:26 +05:30
Harshit Bansal ec1ce08523 emoji_setup_utils: Migrate to python3 type annotations. 2018-07-13 21:19:26 +05:30
Harshit Bansal f6fe5d6266 build_emoji: Migrate to python3 type annotations. 2018-07-13 21:19:25 +05:30
Harshit Bansal f636882e04 build_emoji: Migrate to use `emoji_names.py` file.
This migrates Zulip to use a dramatically better set of names and
aliases for our emoji set, defined in emoji_names.py (which is in turn
manually generated from our hand-curated CSV file).

This should significantly improve the experience of using Zulip's
emoji picker and emoji typeahead for finding what one is looking for.
2018-07-13 21:18:02 +05:30
Tim Abbott 7a86e83a6f build_emoji: Use with for writing to files.
This is a nice code cleanup.
2018-07-13 21:07:08 +05:30
Tim Abbott 85243c525d build_emoji: Pass around emoji_map data structure less.
It wasn't used in either of these functions.
2018-07-13 21:04:37 +05:30
Harshit Bansal b9f974ba61 tools: Add a tool to validate emoji name listing scripts.
This tools emoji name import/export tooling by verifying that no
information gets modified in a round-trip conversion.
2018-07-13 20:53:25 +05:30
Harshit Bansal 628e868d1e tools: Add `export_emoji_names_to_csv`.
This tool is used for exporting `emoji_names.py` to a csv file.
2018-07-13 20:53:08 +05:30
Harshit Bansal 05f85eb94d emoji: Add emoji_names.py file.
Credits to @rishig, Alice Lai, and @rntharu for naming all the emoji.

Names are inspired by iamcal, gemoji, and unicode names, sources like
emojipedia and iemoji, google search results for articles about emoji,
and emoji usage on twitter.
2018-07-13 20:53:07 +05:30
Harshit Bansal ae17a15b29 tools: Add generate_emoji_names_table tool.
This tool generates a listing of the emojis as per the current cache
in use.
2018-07-13 20:53:07 +05:30
Harshit Bansal fef5b43e22 minor: Fix directory name label in 'import_emoji_names_from_csv'. 2018-07-13 20:53:02 +05:30
Tim Abbott ab5fef87e5 models: Document RealmAuditLog. 2018-07-13 18:38:33 +05:30
Max Nussenbaum 122dcc9760 billing: Style upgrade page.
This styles /upgrade.
2018-07-13 18:32:29 +05:30
Max Nussenbaum 8fd900626d billing: Style billing page.
This adds styles to /billing.
2018-07-13 18:30:38 +05:30
Max Nussenbaum 4af3b8bdda settings: Improve styling of draggable profile field rows.
This changes the icon indicating that you can drug the custom
profile field rows in settings to be the double rows of dots
typically used to indicate draggability. It also gives those
rows the "move" cursor on hover.
2018-07-13 18:17:03 +05:30
Cynthia Lin 960f83238b help: Replace perfect-scrollbar with simplebar in help pages.
The perfect-scrollbar library created one major problem, which is that
`ctrl-F` didn't work, and several smaller problems.

Fixes #9748 and fixes #8887.
2018-07-13 18:03:32 +05:30
Cynthia Lin ab9f167053 help: Refactor help page CSS to eliminate padding on content containers.
This is essential for using simplebar, since simplebar doesn't account
for parent <div> paddings, which might cause scrollbars to be mispositioned
if not considered.
2018-07-13 18:03:32 +05:30
Cynthia Lin 1babd6aa28 help: Prevent scrolling on sidebar when collapsed in mobile view. 2018-07-13 18:03:32 +05:30
Cynthia Lin 8ee034a41c help: Prevent scrolling when cursor is on body.
When you hover above the navbar, the cursor focuses on the page
body and scrolls the entire page, breaking the positioning of the
fixed sidebar and page content.

We disable scrolling on the body but allow the sidebar and Markdown
content page elements to be scrolled to fix this bug.
2018-07-13 18:03:32 +05:30
Tim Abbott 3cc93fd721 setup_venv: Fix missing libssl-dev dependency.
We were already correctly including libssl-dev in Zulip's dependencies
in development environment provisioning, but (at least now) it's
needed to build certain Python packages like pycurl when building a
Zulip virtualenv in production.  I haven't investigated why we didn't
need this on Ubuntu, but one possible reason would be that some other
library in our dependencies list happens to depend on it on Ubuntu.

We fix this by moving the dependency over to the shared
VENV_DEPENDENCIES list.

Fixes part of #9946.
2018-07-13 18:00:38 +05:30
Tim Abbott 8f9b5633b8 zulip_tools: Fix accessing LSB data on Debian stretch.
Apparently, at least some Debian stretch systems don't have an
/etc/lsb-release, so the optimization that we did in
5d39a0f0fc broke our installer on
Debian.

We fix this, by falling back to calling the lsb_release command on
systems that don't have a faster way to do it.

Fixes part of #9946.
2018-07-13 18:00:38 +05:30
Aastha Gupta dfde4fac85 invitations: Send 'invites_changed' event for invitations events.
Fixes #7665

In case of invitation events, 'invites_changed' event without
any real payload is sent to all the realm admins and the user.
The event is handled by reloading the list to view recent changes.

Commit tweaked by shubhamdhama:
* Send an `invite_changed` event when an user accept an invite.
    Also, added the test for the same.
* No need to delete the invite list in frontend, current logic
    handles the case when the invite data is changed properly.
* Extracted the common logic for sending an event into
    `notify_invites_changed`.
2018-07-13 18:00:06 +05:30
Tim Abbott 40efac2193 api docs: Clean of high levels of code nesting.
An early continue here makes the code a lot more readable.
2018-07-13 17:39:20 +05:30
Yago González 1cdd6862b7 api docs: Display example arguments as JSON.
Some of the arguments in our REST API have to be sent as JSON objects,
which only accept double quotes for strings.

If we display the examples as normal Python objects, the syntax would be
quite similar but it would use simple quotes, which is invalid JSON (and
isn't accepted by the server).

That's why all the examples should be JSON-serialized in order to comply
with the API's requirements.
2018-07-13 17:36:41 +05:30
Yago González c2f3c4282b api docs: Show message when there are no arguments.
Until now, we were displaying an empty "Arguments" section in the REST
API docs whenever an endpoint didn't use input arguments.

In the case of OpenAPI-based docs, that was also annoying because it
required removing the {generate_api_arguments_table|...} template tag or
leaving an empty "parameters" field in zulip.yaml.

After this, we show a paragraph indicating that the endpoint doesn't
need arguments under the "Arguments" section.
2018-07-13 17:36:40 +05:30
Yago González e6631db6b6 api docs: Raise exception on missing argument file.
If the argument table generator isn't able to reach a file that is
supposed to read, the two most likely causes are:

- The source .md documentation file that is requesting the table has a
typo in the path.
- The file with the arguments isn't there, for some reason.

In either case, we don't want the server to fail silently-ish and
display the docs as if there was no arguments for that endpoint. That's
why the most logic thing to do is to raise an exception and let the
admins know that there's something wrong.
2018-07-13 17:33:06 +05:30
Anders Kaseorg f75c0e29f4 HTML validation: Remove obsolete attributes from <script>, <style>.
<script charset=…>, <script type=…>, and <style type=…> are “obsolete
but conforming” in HTML5.  They make the validator.nu output noisier
and real problems a little harder to find.

(type was required in HTML 4, which is not relevant to us.)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-13 10:27:23 +05:30
Anders Kaseorg c28ab72124 HTML validation: Remove invalid attributes.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-13 10:27:23 +05:30
Cynthia Lin 0ed13e3631 stats: Fix incorrect data-user attribute on Everyone button.
For some reason, it was defined as "realm" instead of "everyone",
resulting in undefined value JS errors when draw_plot() was called.

Fixes #9937.
2018-07-13 10:24:13 +05:30
Rhea Parekh fff0f64dea tests: Refactor tests for importing huddles. 2018-07-13 01:14:42 +05:30
Rhea Parekh ee830ddee9 tests: Fix mypy bug in test_import_realm. 2018-07-13 01:14:42 +05:30
Yago González 0e135b69f9 alert words: Return the updated list when adding/removing words.
POST and DELETE operations in /users/me/alert_words may leave the
user's list of alert words in an unknown state: POSTing adds words to a
list that the client may not know from the begining, and the same with
DELETE.

Replying with the current status of the alert words list is the best way
of letting the client alter the list and knowing its contents after
being updated with a single query.

This is especially useful taking into account that POSTing words that
were already present and DELETing non-existing words both produce a
successful response.

An extra test has been added to avoid leaving GET /users/me/alert_words
too untested.
2018-07-13 01:01:00 +05:30
Yago González 3b21b17a35 alert words: Make alert_words required for POST and DELETE.
Querying an endpoint with no information (thus a noop) and it producing
a successful response doesn't seem to be expected.

Given the case that the client makes such query with no content it will
probably be unintentional and the API should let them know about it.
2018-07-13 01:01:00 +05:30
Joshua Pan 3ac49c1503 mlv: Remove unecessary function names.
We have deprecated this code style.
2018-07-12 11:43:21 -04:00
Joshua Schmidlkofer b1a57d144f thumbor: Add production installer/puppet support.
This commits adds the necessary puppet configuration and
installer/upgrade code for installing and managing the thumbor service
in production.  This configuration is gated by the 'thumbor.pp'
manifest being enabled (which is not yet the default), and so this
commit should have no effect in a default Zulip production environment
(or in the long term, in any Zulip production server that isn't using
thumbor).

Credit for this effort is shared by @TigorC (who initiated the work on
this project), @joshland (who did a great deal of work on this and got
it working during PyCon 2017) and @adnrs96, who completed the work.
2018-07-12 20:37:34 +05:30
Rhea Parekh e9884916c9 import: Support import of huddles.
For importing huddles we have to have unique huddle hashes.
Huddle hashes are extracted from the list of users participating
in a huddle. So to extract these user ids, we first use huddle
id to getting the matching recipient, and then we use subscription
to get the user ids from the recipient id.

Added tests for the same (tests slightly tweaked by tabbott).
2018-07-12 19:06:52 +05:30
Yago González fa1a695e08 test-api: Fix broken check for get_members.
The tests for GET /users were looking for a specific user, asuming that
it would always be in the same position. Since the users' sorting isn't
guaranteed in any way, this can lead to errors in the tests.

Now we make sure the user we grab from the list is the one we need by
checking its email address.

This is just a hotfix that addresses the short-term problem: we have
already made some efforts to make sure these tests are more
deterministic, and now we only need to finish the migration of the old
enpoints to the new system as a long-term solution.
2018-07-12 18:54:06 +05:30