Commit Graph

9771 Commits

Author SHA1 Message Date
Vishnu Ks 9273d0e7be tests: Check for private message special case in consented realm export. 2019-05-21 14:10:29 -07:00
Vishnu Ks 0a466914c4 tests Add PM tests for test_export_realm_with_exportable_user_ids. 2019-05-21 14:10:29 -07:00
Vishnu Ks 044d620ee0 tests Add PM tests for test_zulip_realm. 2019-05-21 14:10:29 -07:00
Vishnu Ks 5a8ddffb0b tests: Split export realm with exportable_user_ids into seperate test. 2019-05-21 14:10:29 -07:00
Vishnu Ks 3f24cc4b86 tests: Move find_by_id to ZulipTestCase. 2019-05-21 14:10:29 -07:00
Vishnu Ks 68387b9f00 tests: Move get_set to ZulipTestCase. 2019-05-21 14:10:29 -07:00
Vishnu Ks cc88f95229 tests: Test for PMs in test_export_realm_with_member_consent.
Sending PM from a hamlet(consented) to othello is a case
of sending message from a consented user to a non consented
user. This result in the generation of more than one message
files during realm export. To handle this case _export_realm
is updated.
2019-05-21 14:10:29 -07:00
Vishnu Ks 21e7763886 export: Remove unnecessary query from export_partial_message_files.
The query is not required anymore after the refactoring done
while merging #12225.
2019-05-21 14:10:29 -07:00
Vishnu Ks 319fe62d44 test: Fix third huddle variable name in test_import_export. 2019-05-21 14:10:29 -07:00
Tim Abbott bde9b28589 test_retention: Update debugging code for CI failures.
This should provide more helpful output for the next stage of
debugging.
2019-05-21 14:10:15 -07:00
Tim Abbott 3de3cb7385 tests: Remove accidentally added empty test file. 2019-05-21 10:03:01 -07:00
Tim Abbott 55b15ba117 test_retention: Improve and extent print-debugging.
We needed flush=True to have output not be lost.

Also print the original messages, so we can compare what's missing.
2019-05-21 09:28:03 -07:00
Tim Abbott 3996b62c79 test_messages: Fix logging in as a bot user.
Bot users can't login, but apparently one could force it with the
Django test system's login function.  We should just post to the API.
2019-05-20 20:21:35 -07:00
Wyatt Hoodes 4dd8c133a9 export: Rename `--upload-to-s3` to be `--upload`.
The upload option will no longer be limited to strictly S3 uploads. This
commit serves as a preliminary step for supporting LOCAL_UPLOADS_DIR as
part of the public only export feature.
2019-05-20 19:59:57 -07:00
Tim Abbott 1353e94b29 test_retention: Add print-debugging.
We've been seeing nondeterministic failures in this test suite in CI
that we can't reproduce locally; these print statements should help
track them down.
2019-05-20 19:43:28 -07:00
Mateusz Mandera 4f9bbe9d66 email_mirror: Clean up and expand test_get_missed_message_token.
This is the only function in TestEmailMirrorLibrary, so we rename this
class to more appropriate TestGetMissedMessageToken, clean it up a bit
and add some extra checks to finally get email_mirror.py to 100% test
coverage.
2019-05-20 19:35:32 -07:00
Mateusz Mandera a0efd76f4e email_mirror: Rewrite log_and_report and cover it with tests.
log_and_report and its helper functions were mostly old code no longer
well adapted to how email mirror works currently, as well as having no
test coverage. We rewrite this part of the email to report errors in a
similar manner, and add tests for it. We're able to get rid of the
clunky and now useless debug_info dictionary in process message, as
log_and_report only needs the recipient email in its third argument.
2019-05-20 19:35:32 -07:00
Mateusz Mandera 2adcdd0c25 email_mirror: Don't pass debug_info to process_stream_message.
The only place in which process_stream_message used debug_info was to
set the 'stream' key, which would only be used if ZulipEmailForwardError
was raised after this line in the code - which is impossible, because after
that line only send_zulip (which doesnt raise this exception) and
logger.info get called, then process_stream_message successfully returns
and then process_message succesfully returns as well. So this debug_info
code wasn't doing anything. We remove it.
2019-05-20 19:35:32 -07:00
Aditya Bansal 43591d6c71 archives: Display globe icon for web public streams.
In this commit we start reserving the globe icon for web public
streams and replace instances of mixed usage of globe for public
streams with a '#'.
2019-05-20 18:13:31 -07:00
Joshua Pan c28c301506 org_settings: Display delivery_email to admins always is admin UI.
Mostly rewritten by Tim Abbott to ensure it correctly implements the
desired security model.

Administrators should have access to users' real email address so that
they can contact users out-of-band.
2019-05-20 18:13:31 -07:00
Tim Abbott 07856ad648 api: Override client_gravatar for EMAIL_ADDRESS_VISIBILITY_ADMINS.
Clients won't have access to user email addresses, and thus won't be
able to compute gravatars.

The tests for this are a bit messy, in large part because our tests
for get_events call subsections of it, rather than the main function.
2019-05-20 18:13:31 -07:00
Tim Abbott bcc6949461 zilencer: Add better error handling for IntegrityError.
This provides a clean warning and 40x error, rather than a 500, for
this corner case which is very likely user error.

The test here is awkward because we have to work around
https://github.com/zulip/zulip/issues/12362.
2019-05-20 17:53:43 -07:00
Rishi Gupta fbc2239f73 keyboard shortcuts: Update description for S. 2019-05-20 15:10:11 -07:00
Puneeth Chaganti bdf8183f55 open graph: Uploaded realm icon URL is absolute for some backends.
The `LocalUploadBackend` returns a relative URL, while the `S3UploadBackend`
returns an absolute URL. This commit switches to using `urljoin` to obtain the
absolute URL, instead of simply joining strings.
2019-05-20 13:14:28 -07:00
Hemanth V. Alluri 1a12e112d9 devtools: Completely overhaul the frontend for the integrations devtool.
This commit also adds a small functionality change where the results of
each webhook fixture message sent is now displayed to the user.

With a small tweak by tabbott to fix a styling bug.

Fixes #12122.
2019-05-20 12:36:15 -07:00
Hemanth V. Alluri ef98211f68 devtools: Add support for non-json fixtures for the integrations tool.
Note: If you're going to send fixtures which are not JSON or of the
text/plain content type, make sure you set the correct content type
in the custom headers.

E.g. For the wordpress fixtures the "Content-Type" should be set to
"application/x-www-form-urlencoded".
2019-05-20 12:31:52 -07:00
Hemanth V. Alluri 8214d65336 devtools: Add support for send_all in the integrations dev panel.
Using this feature a reviewer can easily send and view all fixture
messages for any given integration - with only JSON fixtures that is.
2019-05-20 12:29:10 -07:00
K.Kanakhin e930851d16 retention-period: Add more core code for retention policy.
This is a very old commit for #106, which has been on hiatus for a few
years.  It was significantly modified by tabbott to:
* Improve coding style and variable names
* Update mypy annotations style
* Clean up the testing logic
* Update for API changes elsewhere in our system

But the actual runtime code is essentially unmodified from the
original work by Kirill.

It contains basic support for archiving Messages, UserMessages, and
Attachments with a nice test suite.  It's still not usable in
production (e.g. it will probably break Reactions, SubMessages, etc.),
but upcoming commits will address that.
2019-05-19 20:22:47 -07:00
Tim Abbott ec4221abb9 actions: Return message ID from internal_send_*_message.
This is handy for code that needs to do something with the sent
message.  We need it for a retention policy code path, but it seems
likely we'll use it a lot down the line.
2019-05-19 19:18:06 -07:00
Hemanth V. Alluri 2bd9c8cb42 devtools: Add custom HTTP headers support to the integrations dev panel.
This commit introduces a simple field where the user can now specify custom
HTTP headers. This commit does not introduce an improved system for storing
HTTP headers as fixtures - such a change would modify both the existing unit
tests as well as this devtool.
2019-05-17 11:50:53 -07:00
Hemanth V. Alluri 64b4fd5923 management: Extract parse_headers in send_webhook_fixture_message.
This way other devtools such as the integrations development panel
could also use the method.
2019-05-17 11:50:53 -07:00
Puneeth Chaganti 5f50c4d799 cache: Clear realm descriptions when description is changed.
Clear both the rendered and the text descriptions for a realm, when the
description of the realm is changed.
2019-05-17 11:31:14 -07:00
Puneeth Chaganti f084075c2a cache: Clear rendered realm text description when flushing realm.
The commit 2d9c5b3575 introduced this cache key,
and didn't setup clearing of the cache correctly.
2019-05-17 11:31:14 -07:00
Anders Kaseorg cdd09d0441 meta_tags: Remove twitter:title, twitter:description, twitter:image.
Twitter falls back to og:title, og:description, og:image, which we set
identically, so these are redundant.

https://developer.twitter.com/en/docs/tweets/optimize-with-cards/overview/markup.html

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-16 21:19:08 -07:00
Hemanth V. Alluri bae8295c52 devtools: Add integrations dev panel.
This commit adds a new developer tool: The "integrations dev panel"
which will serve as a replacement for the send_webhook_fixture_message
management command as a way to test integrations with much greater ease.
2019-05-15 13:07:44 -07:00
Vishnu Ks 06983298ba export: Add support for exporting realm with member consent.
This lets us handle directly in our tooling the user experience that
we document for exporting a realm with member consent (before, it
required unpleasant manual work).
2019-05-15 12:35:32 -07:00
Vishnu Ks 8ebdbea4d5 tests: Remove commented out print statements in test_import_export. 2019-05-15 11:57:31 -07:00
vinitS101 611f1f8fd2 left_sidebar: Add "+Add streams" to bottom of streamlist.
Added a new button at the bottom of the stream list which redirects
users to '/#streams/all' where they can create new streams or subscribe
to new streams.
The button is not visible to guests.

Fixes #11642.
2019-05-14 16:12:51 -07:00
Tim Abbott dc2aa031f6 linkifiers: Allow semicolons in target URLs.
This fixes another user report.  The right fix is to convert this to
use a normal URL validator after substitution.
2019-05-14 11:17:55 -07:00
Puneeth Chaganti 4ac9778d69 url preview: Catch network errors during get for page content.
We may be successfully able to get the page once, to get the content type, but
the server or network may go down and cause problems when fetching the page for
parsing its meta tags.
2019-05-13 13:55:00 -07:00
Puneeth Chaganti 9fd1c40bb1 url preview: Timeout requests after 15 seconds. 2019-05-13 13:54:59 -07:00
Puneeth Chaganti 0b76b16101 url preview: Set a custom user agent for requests.
Some sites seem to block the default user agent of the requests
library. Using a custom user agent lets us show previews for some of
these sites.
2019-05-13 13:54:43 -07:00
Puneeth Chaganti 59555ee7e5 url preview: Confirm content-type before trying to show previews.
Currently, we only show previews for URLs which are HTML pages, which could
contain other media. We don't show previews for links to non-HTML pages, like
pdf documents or audio/video files. To verify that the URL posted is an HTML
page, we verify the content-type of the page, either using server headers or by
sniffing the content.

Closes #8358
2019-05-13 13:45:17 -07:00
Tim Abbott 0f2d7a354f realm_filters: Allows more use of & and friends in URLs.
We had some excessively tight rules about what characters were
allowed, which in particular prevented using `?foo=bar&baz=quux`
structures in the realm filters URLs.

Fixes #12239.
2019-05-13 13:31:09 -07:00
Yashashvi Dave 6f6e87f45d models: Rename 'Jitsi' to 'Jitsi Meet' in Realm model.
Fixes #12293
2019-05-13 12:08:04 -07:00
Yashashvi Dave 3efead6f5f /json/realm: Return error on not-avaliable video chat provider value.
This adds some missing error handling from the video chat provider
code path.
2019-05-13 12:07:44 -07:00
Yashashvi Dave cb85ca8601 models: Alter video_chat_provider field type to integer.
Migration rewritten by tabbott because it did not work.
2019-05-13 12:02:28 -07:00
Yashashvi Dave d7f9b7eea3 models: Add dict for video chat providers in Realm model. 2019-05-13 11:34:34 -07:00
Yashashvi Dave bbff797a93 zerver/tests/test_realm: Fix not-avaliable video-chat-provider value. 2019-05-13 11:34:34 -07:00
Tim Abbott 45305c93ae auth: Fix Apache SSO port number in confirmation links.
We had a report in the thread around
https://chat.zulip.org/#narrow/stream/31-production-help/topic/Apache-based.20SSO/near/741013
that confirmation links were taking the user to the /register form on
the Apache server, which of course doesn't work because the Apache
server architecture we have is intended to only serve a single
endpoint, /accounts/login/sso, and not any static assets (etc.).

This manifested as users getting a broke page with a bunch of JS
errors about missing static assets when trying to sign up for an
account.  The right fix is to ensure that we serve these confirmation
links (and maybe in the future, redirects) to the nginx server.
2019-05-13 11:14:41 -07:00