Commit Graph

27248 Commits

Author SHA1 Message Date
Tim Abbott 85d7ddbeca thumbor: Switch thumbor.conf to use a better system.
Previously, thumbor.conf read LOCAL_UPLOADS_DIR from
zproject/settings.py by calling get-django-setting, which ends up
importing Django (taking 250ms, and requiring a full Zulip virtualenv
to be installed on the system).

The new system (in production) just imports prod_settings.py, which
has a lot fewer dependencies and is also a lot faster.
2018-07-15 00:05:05 +05:30
Aditya Bansal 0e50afb599 thumbor: Add ldap and django as a dependancy.
We add this dependancy to thumbor for no use other than making an
import possible in one of the upcoming commits. Basically we wanted to
import LOCAL_UPLOADS_DIR from zproject.prod_settings or
zproject.dev_settings and prod_settings_template.py imports
django-auth-ldap (which depends on python-ldap and django).

This seems counterproductive, but it makes it possible for us to save
significant thumbor server startup time that would have been consumed
in `get-django-setting`, and once thumbor supports Python 3, we'll
probably be merging the virtualenvs anyway (in which case this change
would become a no-op).
2018-07-15 00:03:21 +05:30
Tim Abbott 52f5d83c90 thumbor: Add libthumbor dependency.
We need this for signing thumbnail requests from within Zulip.
2018-07-14 21:46:02 +05:30
Tim Abbott 50588274ff zthumbor: Use get_secret to configure thumbor.conf.
We need to use the value of SECURITY_KEY read from
/etc/zulip/zulip-secrets.conf (using get_secret), and also we disable
a few Thumbor features that we're not using. We also fix access to
secret keys by getting their values using get_secret.
2018-07-14 21:45:33 +05:30
Sarah de2ec8a05c notification settings: Add UI for stream email notifications.
Previous commits have fully implemented the logic for stream email
notifications; this final commit adds support for configuring it to
the UI.

Fixes #6511.
2018-07-14 12:19:33 +05:30
Sarah de947445ca event_queue: Add stream_email_notify.
This adds support to the event queue system for triggering
missed-message notifications (whether push or email) to support the
stream push notifications feature.
2018-07-14 12:19:33 +05:30
Sarah 73f5be65e8 zerver/lib/notifications: Add stream email notifications.
This modifies the logic for formatting outgoing missed-message emails
to support the upcoming stream email notifications feature (providing
a new format for the subject, etc.).
2018-07-14 12:19:33 +05:30
Tim Abbott 47df8c0c6e notifications: Elimininate now-unnecessary database query.
Because we're passing through the trigger for notifications to
do_send_missedmessage_events_reply_in_zulip, we don't need to go back
to the database to determine which messages actually mentioned the
user.
2018-07-14 12:19:33 +05:30
Tim Abbott dabf3667d1 notifications: Eliminate unnecessary at_mention context variable.
we already had a variable "mention" that did the exact same thing.
2018-07-14 12:19:33 +05:30
Tim Abbott bc861b4441 notifications: Pass trigger to do_send_missedmessage_events_reply_in_zulip.
This change converts our logic for determining whether the current
user was mentioned in a group of messages from the implicit "if it was
sent to a stream, it's a mention" to the explicit "we actually know
there was a mention in the message".  This is an important
prerequisite for our upcoming feature to support getting email
notifications for streams always (even without a mention).
2018-07-14 12:02:08 +05:30
Tim Abbott 58a7a390c8 event_queue: Call build_offline_notification unconditionally.
Previously, maybe_enqueue_notifications had this very subtle logic,
where it set the notice variable only inside the block for push
notifications, but then also used it inside the block for email
notifications.

This "worked", because previously the conditions for push
notifications were always true if the conditions for email
notifications were, but the code was unnecessarily confusing.  The
only good reason to write it this way is if build_offline_notification
was expensive; in fact, the most expensive thing it does is calling
time.time(), so that reason does not apply here.

This was further confusing, in that in the original logic, we relied
on the fact that push notification code path edited the "notice"
dictionary for further processing.

Instead, we just call it separately and setup the data separately in
each code path.
2018-07-14 12:01:59 +05:30
Tim Abbott a09ebf0551 event_queue: Remove confusing comment about rabbitmq.
Whatever RabbitMQ check this comment used to be next to, it isn't next
to anymore.
2018-07-14 12:00:37 +05:30
Tim Abbott 08ed1b4605 notifications: Set trigger on offline email notifications.
This data will be required for correctly implementing the upcoming
stream_push_notify feature; it also helps support cleaning up the code
for the existing stream mentions logic.
2018-07-14 11:57:28 +05:30
Tim Abbott b0c393cf97 notifications: Format missed_messages argument as a dict.
Because in upcoming commits, we'll want to pass additional per-message
data into do_send_missedmessage_events_reply_in_zulip, we need to
expand the format for how we represent messages to account for that.
2018-07-14 11:34:04 +05:30
Aditya Bansal 2e837b1407 archives: Change the topic/<topic_name> code path to use topics prefix.
We do this for maintaining consistency in naming of the endpoints.
2018-07-14 10:39:35 +05:30
Tim Abbott bf1ad714da actions.py: Refactor generate_topic_history_from_db_rows for clarity.
This refactors the generate_topic_history_from_db_rows function to not
depend upon the assumption of rows passed as parameter to be sorted in
reverse order of max_message_id field.

Additionally, we add sorting and some tests that verify correct
handling of these cases.
2018-07-14 10:11:08 +05:30
Aditya Bansal 63ec8b08b8 archives: Add endpoint to fetch topic history of web public streams.
In this commit we add a new endpoint so as to have a way of fetching
topic history for a given stream id without having to be logged in.
This can only happen if the said stream is web public otherwise we
just return an empty topics list. This endpoint is quite analogous
to get_topics_backend which is used by our main web app.

In this commit we also do a bit of duplication regarding the query
responsible for fetching all the topics from DB. Basically this
query is exactly the same as what we have in the
get_topic_history_for_stream function in actions.py. Basically
duplicating now is the right thing to do because this query is
really gonna change when we add another criteria for filtering
messages which is:
Only topics for messages which were sent during the period the
corresponding stream was web public should be returned.
Now when we will do this, the query will change and thus it won't
really be a code duplication!
2018-07-14 09:51:37 +05:30
Eeshan Garg 37e7cb4b96 outgoing-webhooks.md: Use HTML tables to document payload fields. 2018-07-14 09:44:54 +05:30
Eeshan Garg 2e78a56796 outgoing-webhooks.md: Add example payloads/fixtures. 2018-07-14 09:44:54 +05:30
Eeshan Garg 67045d65a9 webhooks/gitlab: Include title in Issue Hook event messages.
We already include the issue title in the topic. But if one chooses
to group all gitlab notifications under one topic, the message body
is misleading in the sense that only the Issue ID and the description
are displayed, not the title, which isn't super helpful if the topic
doesn't tell you the title either.

I think we should err on the side of always including the title in
the main message body, which is what this commit does.

Fixes #9913.
2018-07-14 09:38:11 +05:30
Tim Abbott 811ac718d6 build_emoji: Document a quirk of generate_sprite_css_files. 2018-07-13 21:19:26 +05:30
Harshit Bansal ec1ce08523 emoji_setup_utils: Migrate to python3 type annotations. 2018-07-13 21:19:26 +05:30
Harshit Bansal f6fe5d6266 build_emoji: Migrate to python3 type annotations. 2018-07-13 21:19:25 +05:30
Harshit Bansal f636882e04 build_emoji: Migrate to use `emoji_names.py` file.
This migrates Zulip to use a dramatically better set of names and
aliases for our emoji set, defined in emoji_names.py (which is in turn
manually generated from our hand-curated CSV file).

This should significantly improve the experience of using Zulip's
emoji picker and emoji typeahead for finding what one is looking for.
2018-07-13 21:18:02 +05:30
Tim Abbott 7a86e83a6f build_emoji: Use with for writing to files.
This is a nice code cleanup.
2018-07-13 21:07:08 +05:30
Tim Abbott 85243c525d build_emoji: Pass around emoji_map data structure less.
It wasn't used in either of these functions.
2018-07-13 21:04:37 +05:30
Harshit Bansal b9f974ba61 tools: Add a tool to validate emoji name listing scripts.
This tools emoji name import/export tooling by verifying that no
information gets modified in a round-trip conversion.
2018-07-13 20:53:25 +05:30
Harshit Bansal 628e868d1e tools: Add `export_emoji_names_to_csv`.
This tool is used for exporting `emoji_names.py` to a csv file.
2018-07-13 20:53:08 +05:30
Harshit Bansal 05f85eb94d emoji: Add emoji_names.py file.
Credits to @rishig, Alice Lai, and @rntharu for naming all the emoji.

Names are inspired by iamcal, gemoji, and unicode names, sources like
emojipedia and iemoji, google search results for articles about emoji,
and emoji usage on twitter.
2018-07-13 20:53:07 +05:30
Harshit Bansal ae17a15b29 tools: Add generate_emoji_names_table tool.
This tool generates a listing of the emojis as per the current cache
in use.
2018-07-13 20:53:07 +05:30
Harshit Bansal fef5b43e22 minor: Fix directory name label in 'import_emoji_names_from_csv'. 2018-07-13 20:53:02 +05:30
Tim Abbott ab5fef87e5 models: Document RealmAuditLog. 2018-07-13 18:38:33 +05:30
Max Nussenbaum 122dcc9760 billing: Style upgrade page.
This styles /upgrade.
2018-07-13 18:32:29 +05:30
Max Nussenbaum 8fd900626d billing: Style billing page.
This adds styles to /billing.
2018-07-13 18:30:38 +05:30
Max Nussenbaum 4af3b8bdda settings: Improve styling of draggable profile field rows.
This changes the icon indicating that you can drug the custom
profile field rows in settings to be the double rows of dots
typically used to indicate draggability. It also gives those
rows the "move" cursor on hover.
2018-07-13 18:17:03 +05:30
Cynthia Lin 960f83238b help: Replace perfect-scrollbar with simplebar in help pages.
The perfect-scrollbar library created one major problem, which is that
`ctrl-F` didn't work, and several smaller problems.

Fixes #9748 and fixes #8887.
2018-07-13 18:03:32 +05:30
Cynthia Lin ab9f167053 help: Refactor help page CSS to eliminate padding on content containers.
This is essential for using simplebar, since simplebar doesn't account
for parent <div> paddings, which might cause scrollbars to be mispositioned
if not considered.
2018-07-13 18:03:32 +05:30
Cynthia Lin 1babd6aa28 help: Prevent scrolling on sidebar when collapsed in mobile view. 2018-07-13 18:03:32 +05:30
Cynthia Lin 8ee034a41c help: Prevent scrolling when cursor is on body.
When you hover above the navbar, the cursor focuses on the page
body and scrolls the entire page, breaking the positioning of the
fixed sidebar and page content.

We disable scrolling on the body but allow the sidebar and Markdown
content page elements to be scrolled to fix this bug.
2018-07-13 18:03:32 +05:30
Tim Abbott 3cc93fd721 setup_venv: Fix missing libssl-dev dependency.
We were already correctly including libssl-dev in Zulip's dependencies
in development environment provisioning, but (at least now) it's
needed to build certain Python packages like pycurl when building a
Zulip virtualenv in production.  I haven't investigated why we didn't
need this on Ubuntu, but one possible reason would be that some other
library in our dependencies list happens to depend on it on Ubuntu.

We fix this by moving the dependency over to the shared
VENV_DEPENDENCIES list.

Fixes part of #9946.
2018-07-13 18:00:38 +05:30
Tim Abbott 8f9b5633b8 zulip_tools: Fix accessing LSB data on Debian stretch.
Apparently, at least some Debian stretch systems don't have an
/etc/lsb-release, so the optimization that we did in
5d39a0f0fc broke our installer on
Debian.

We fix this, by falling back to calling the lsb_release command on
systems that don't have a faster way to do it.

Fixes part of #9946.
2018-07-13 18:00:38 +05:30
Aastha Gupta dfde4fac85 invitations: Send 'invites_changed' event for invitations events.
Fixes #7665

In case of invitation events, 'invites_changed' event without
any real payload is sent to all the realm admins and the user.
The event is handled by reloading the list to view recent changes.

Commit tweaked by shubhamdhama:
* Send an `invite_changed` event when an user accept an invite.
    Also, added the test for the same.
* No need to delete the invite list in frontend, current logic
    handles the case when the invite data is changed properly.
* Extracted the common logic for sending an event into
    `notify_invites_changed`.
2018-07-13 18:00:06 +05:30
Tim Abbott 40efac2193 api docs: Clean of high levels of code nesting.
An early continue here makes the code a lot more readable.
2018-07-13 17:39:20 +05:30
Yago González 1cdd6862b7 api docs: Display example arguments as JSON.
Some of the arguments in our REST API have to be sent as JSON objects,
which only accept double quotes for strings.

If we display the examples as normal Python objects, the syntax would be
quite similar but it would use simple quotes, which is invalid JSON (and
isn't accepted by the server).

That's why all the examples should be JSON-serialized in order to comply
with the API's requirements.
2018-07-13 17:36:41 +05:30
Yago González c2f3c4282b api docs: Show message when there are no arguments.
Until now, we were displaying an empty "Arguments" section in the REST
API docs whenever an endpoint didn't use input arguments.

In the case of OpenAPI-based docs, that was also annoying because it
required removing the {generate_api_arguments_table|...} template tag or
leaving an empty "parameters" field in zulip.yaml.

After this, we show a paragraph indicating that the endpoint doesn't
need arguments under the "Arguments" section.
2018-07-13 17:36:40 +05:30
Yago González e6631db6b6 api docs: Raise exception on missing argument file.
If the argument table generator isn't able to reach a file that is
supposed to read, the two most likely causes are:

- The source .md documentation file that is requesting the table has a
typo in the path.
- The file with the arguments isn't there, for some reason.

In either case, we don't want the server to fail silently-ish and
display the docs as if there was no arguments for that endpoint. That's
why the most logic thing to do is to raise an exception and let the
admins know that there's something wrong.
2018-07-13 17:33:06 +05:30
Anders Kaseorg f75c0e29f4 HTML validation: Remove obsolete attributes from <script>, <style>.
<script charset=…>, <script type=…>, and <style type=…> are “obsolete
but conforming” in HTML5.  They make the validator.nu output noisier
and real problems a little harder to find.

(type was required in HTML 4, which is not relevant to us.)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-13 10:27:23 +05:30
Anders Kaseorg c28ab72124 HTML validation: Remove invalid attributes.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-13 10:27:23 +05:30
Cynthia Lin 0ed13e3631 stats: Fix incorrect data-user attribute on Everyone button.
For some reason, it was defined as "realm" instead of "everyone",
resulting in undefined value JS errors when draw_plot() was called.

Fixes #9937.
2018-07-13 10:24:13 +05:30
Rhea Parekh fff0f64dea tests: Refactor tests for importing huddles. 2018-07-13 01:14:42 +05:30