Commit Graph

31740 Commits

Author SHA1 Message Date
Anders Kaseorg 2c0980e3a6 Vagrantfile: Remove VMmare Fusion provider.
Ubuntu 14.04 is EOL.  Mac users can use the VirtualBox provider (and
now maybe the Docker provider?).  We can add back the VMware Fusion
provider later if someone cares enough to get it working with Ubuntu
18.04.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-29 16:06:33 -07:00
Anders Kaseorg dc11f197a1 Vagrantfile: Update virtualbox image to bionic.
This is a key part of our process to deprecate Ubuntu trusty: migrating
the Zulip development environment on macOS and Window to Ubuntu 
Bionic.

As part of this, we need to remove ubuntu-server, because it now
depends on update-notifier-common.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-29 16:04:13 -07:00
Tim Abbott 0330a5ed82 docs: Use bulleted list for user-facing subsystems.
This makes it a bit more visually obvious that there are more than 3
total things expected below.
2019-05-29 15:52:18 -07:00
Tim Abbott 19666aacbf docs: Update link to FCM. 2019-05-29 15:52:11 -07:00
Tim Abbott fa37c5cee1 docs: Extract a Writing Documentation top-level section.
This should make this easier to find, and also makes "Subsystems" a
bit smaller of a catch-all.
2019-05-29 15:52:11 -07:00
Anders Kaseorg 1a68ef2951 Vagrantfile: Restore idempotency to vagrant provision.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-29 15:38:36 -07:00
Anders Kaseorg 03cfe10da7 coveragerc: Omit blub.
The jedi package exec()s some code in the context of the fake module
blub, causing errors when generating the coverage report.  See
https://github.com/davidhalter/jedi/issues/1122.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-29 15:04:47 -07:00
Tim Abbott 7134a12231 pgroonga: Fix do_batch_update logic for all postgres versions.
Apparently, the fix in 430ed061c1
requires a sufficiently modern postgres version not available in
Xenial.

Fixes #12382.

Patch by Sutou Kouhei.
2019-05-29 14:58:44 -07:00
Wyatt Hoodes 857c16d52d Vagrantfile: Add customization to remove log file.
An unnecessary log file gets generated when running vagrant up for the
first time with the Ubuntu Bionic box.  This looks like it is being
caused upstream by the base box containing a Vagrantfile with a line
that looks something like:

vb.customize [ ..., ..., ..., ..., \
File.join(Dir.pwd, "ubuntu-bionic-18.04-cloudimg-console.log") ]

The line added here negates this.
2019-05-29 14:46:54 -07:00
Tim Abbott 49b261b211 i18n: Update translation data from Transifex. 2019-05-28 17:03:35 -07:00
Eeshan Garg cecea75457 api_docs: Detect missing arguments in curl examples.
This commit adds automated tests that make sure that every curl
example command in our API docs has the '-X (POST|GET)' argument.

Fixes: #11927
2019-05-28 16:53:48 -07:00
Tim Abbott 8339c21637 test-backend: Fix db issues with running two copies in parallel.
Sometimes it's useful to run two copies of test-backend at the same
time.  The problem with doing so is that we need to make sure no two
threads are using the same test database ID.

Previously, this worked only if at most one of those copies was
running in the single-threaded mode, because we used a random database
ID for the single-threaded code path, but the same IDs counting from 0
for the parallel code path.

Fix this, mostly, by generating a random start for the range of IDs
used by the process, and then counting off database IDs starting from
there (both in the parallel and non-paralllel modes).

There's still a very low probability race, see the TODO.
Additionally, there appear to be some other races with running two
copies of test-backend at the same time not related to the database.

See https://github.com/zulip/zulip/issues/12426 for a follow-up issue
that's sorta created by this.
2019-05-28 16:38:45 -07:00
Tim Abbott 095ff28277 test-backend: Avoid dropping/rebuiling zulip_test database.
The test-backend parallel test runner system doesn't actually use the
zulip_test database; instead, it creates its own databases off the
zulip_test_template database.

We were accidentally running `tools/generate_fixtures` even when there
are no changes, because this function is shared with the
tools/lib/test_server.py codebase, which needs us to do the work of
creating a test database for it off the zulip_test_template database.

Fixing this saves about 1.5s / 4s of the runtime of a single test.
2019-05-28 16:38:45 -07:00
Tim Abbott e0afdb675f test_fixtures: Rename and document update_test_databases_if_required.
The previous name was confusing because `generate_fixtures` isn't
actually a good name in the first place.
2019-05-28 16:38:45 -07:00
Tim Abbott 7b6f219663 test-backend: Flush stdout when starting testing.
This makes the output more properly ordered.
2019-05-28 16:38:45 -07:00
Tim Abbott 4442509ec3 node: Disable a test suite that's missing a line of coverage. 2019-05-28 16:36:30 -07:00
Tim Abbott 0c06c64f77 docs: Extend background discussion on LDAP integration. 2019-05-28 13:53:36 -07:00
Vishnu Ks 31151dadbf import: Replace data-user-group-id in rendered_content.
See the data-user-id commit for details.
2019-05-28 12:53:20 -07:00
Vishnu Ks ce1d6044db import: Replace data-stream-id in rendered_content.
See the data-user-id commit for details.
2019-05-28 12:53:20 -07:00
Vishnu Ks cb5b3f347b import: Replace data-user-id in rendered_content with new user id.
Previously, if you exported a Zulip organization and then re-imported
it, we'd end up renumbering the user IDs and all direct foreign key
references to them in the database, but not the data-user-id
references in mentions.  Fix this by parsing the message content and
doing that renumbering.

(Because we import raw markdown, not HTML, from third-party tools,
these changes won't affect data import from slack etc.)

Fixes the high-priority part of #11293.
2019-05-28 12:53:19 -07:00
Vaibhav 61371cbe9a user_info: Add `bot_owner_id` to user info dataset.
Modifies the dict with the user info to include the key `bot_owner_id`
so it can be displayed in the user info popover.

Tests concerned with changing bot owner have been modified to have
number of events=2 because while updating the bot info, two events
are fired -- updating the `realm_bot` and `realm_user` since the
key `bot_owner_id` is a part of realm user info.
2019-05-28 12:41:52 -07:00
Anders Kaseorg 802d3dbbf4 authenticate: Use keyword-only parameters.
Since positional arguments are interpreted differently by different
backends in Django's authentication backend system, it’s safer to
disallow them.

This had been the motivation for previously declaring the parameters
with default values when we were on Python 2, but that was not super
effective because Python has no rule against positional default
arguments and that convention for our authentication backends was
solely enforced by code review.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 23:49:54 -07:00
Anders Kaseorg 082f23a659 authenticate: Remove default values for required parameters.
It is now the caller’s responsibility to check that realm is not None.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 23:47:22 -07:00
Anders Kaseorg 725582850f login_or_register_remote_user: Remove unused invalid_subdomain parameter.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 23:47:22 -07:00
David Wood 2e9e653af1 webhooks/jira: Fix markup conversion of full links.
This commit modifies the regex used when parsing JIRA's full links of
the form `[text|link]` so that if you have two in a message, Zulip
markup conversion doesn't think that the first link extends to the
closing `]` of the second link.
2019-05-27 23:41:11 -07:00
Tim Abbott 1fec51d73c help: Update the discussion of fake email addresses. 2019-05-27 23:23:48 -07:00
Rishi Gupta b09c961ef5 docs: Clarify use of we in user docs subsystem. 2019-05-27 23:21:39 -07:00
Rishi Gupta a2cf66c39e help: Update enable-full-width-display. 2019-05-27 23:21:39 -07:00
Rishi Gupta 0eeae3b6d0 email visibility: Enable feature and add user documentation. 2019-05-27 23:21:39 -07:00
Rishi Gupta ca942b4878 settings: Restructure CSS for fa-question-circle-o.
There are only two of these icons in the codebase, so it is a relatively
safe change.

Making this change to make it easier to add this icon to labels and other
places in settings.
2019-05-27 23:14:26 -07:00
Puneeth Chaganti b9eb2aa8e8 lightbox: Switch to using one argument variation of $(...). 2019-05-27 23:07:07 -07:00
Priyank Patel 3f32ffc4eb compose: Use new ID-based api for sending messages.
This only happens if the realm is not a zephyr realm.

Finishes part of #9474.
2019-05-27 22:58:42 -07:00
Vaibhav eed09527d5 css: Use SCSS nesting in components.scss for `.new-style`. 2019-05-27 22:50:11 -07:00
Vaibhav f6051d8819 css: Use SCSS nesting in components.scss for `.simplebar-track`. 2019-05-27 22:50:11 -07:00
Vaibhav 5cc331b93f css: Cleanup CSS corresponding to `.flex-row .field`.
A `.field` element already has left margin set as 10px and setting
left-margin explicitly for a `.field + .field` equal to the same
value is redundant. This rule is removed.

Margin is set as `10px 10px` which can be equivalently written as
`10px`.
2019-05-27 22:50:11 -07:00
Vaibhav c50206b824 css: Use SCSS nesting in components.scss for `.flex-row`. 2019-05-27 22:50:11 -07:00
Vaibhav 95ab1f238e css: Reorder components.scss so that `.flex-row` are in same place. 2019-05-27 22:50:11 -07:00
Wyatt Hoodes 5c82c52b52 export.py: Clean up redundant import statements.
There existed duplicate import statements for the S3 backend as a
result of the prior refactoring work.
2019-05-27 20:13:56 -07:00
Wyatt Hoodes 46790b3280 public_export.py: Reorder the creation of the RealmAuditLog object.
This reordering was originally made with regard to the delete after
access feature for the public export.  However, this reordering is
more correct overall, i.e., the object should be created before the
event pertaining to the object is sent.
2019-05-27 20:07:28 -07:00
Wyatt Hoodes 55ff497099 test_realm_export.py: Remove unnecessary variable assignments.
The `queue_data` variable is an intermediate step that's unnecessary.
Instead, the values from the queue event are assigned dierectly.

Also, the `worker` variable is not worth an assignment as it is only
referenced a single time per test case.
2019-05-27 20:07:28 -07:00
Wyatt Hoodes d32ce0ed7e test_realm_export.py: Clean up the mocking of `do_export_realm`.
A FileNotFound error was set as the side-effect of the do_export_realm
mock and the DeferredWorker was made to consume the event explicitly.

Previously, the mock of do_export_realm was producing spammy output
as a result of a FileNotFound error coming from the queue processing of
`do_write_stats_file_for_realm_export`.
2019-05-27 20:07:28 -07:00
Wyatt Hoodes d3a39d6a5b test_realm_export.py: Remove the `create_s3_buckets` call.
We don't need to create any s3 buckets before hand in this test
case, as we don't get to a point in the testing flow where this
is necessary.
2019-05-27 20:07:28 -07:00
Wyatt Hoodes 551458e776 test_realm_export.py: Remove setUp method to match standard style.
The setUp method for two basic lines of code was removed to match
the typical style of Zulip's unit tests.
2019-05-27 20:07:28 -07:00
Wyatt Hoodes c0ef6c2fc6 export: Add LOCAL_UPLOADS_DIR support to the export feature.
A unique path was created using the `LOCAL_UPLOADS_DIR` backend, similar
to the code used in `LocalUploadBackend`.  The exported tarball was
copied to the directory, and an nginx url was created to serve the file
publicly.

Tweaked by tabbott to output an actual URL.
2019-05-27 20:06:35 -07:00
Anders Kaseorg f21d3be5fb dev-vagrant-docker: Copy the host user’s uid for the vagrant user.
This allows /srv/zulip to be writable from the guest even if the host
user’s uid is not 1000.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 19:21:51 -07:00
Mohit Gupta a98447b312 bots: Bots can post to announcement-only streams if their owner can.
Bot owned by a non-admin gets blocked but bots owned by an admin
can post to announcement-only stream.

Fixes: #12310.
2019-05-27 18:43:07 -07:00
Mohit Gupta d60f6c9ad9 test_messages: Fix buggy test, bot tests must use /api/v1 endpoint. 2019-05-27 18:43:07 -07:00
Mayank Madan 7fedcbd840 tests: Extract and use assert_logged_in_user_id test helper.
This cleans up the pattern for how we check which user is logged in
during Zulip's backend unit tests to be much more readable (replacing
the arcane session code that does this check).
2019-05-27 18:32:27 -07:00
Anders Kaseorg 90a9dfa7f5 setup-apt-repo: Try to download pre-setup deps before skipping update.
Simulate isn’t enough in some cases.  The error message when this
fails looks sufficiently non-alarming.

LXC:

    default: + apt-get -dy install lsb-release apt-transport-https gnupg
    default: Reading package lists...
    default: Building dependency tree...
    default:
    default: Reading state information...
    default: lsb-release is already the newest version.
    default: gnupg is already the newest version.
    default: The following NEW packages will be installed:
    default:   apt-transport-https
    default: 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
    default: Need to get 25.1 kB of archives.
    default: After this operation, 238 kB of additional disk space will be used.
    default: Err http://archive.ubuntu.com/ubuntu/ trusty-updates/main apt-transport-https amd64 1.0.1ubuntu2.3
    default:   404  Not Found [IP: 91.189.88.161 80]
    default: Err http://security.ubuntu.com/ubuntu/ trusty-security/main apt-transport-https amd64 1.0.1ubuntu2.3
    default:   404  Not Found [IP: 91.189.88.161 80]
    default: E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/a/apt/apt-transport-https_1.0.1ubuntu2.3_amd64.deb  404  Not Found [IP: 91.189.88.161 80]
    default:
    default: E: Some files failed to download
    default: + apt-get update
    […]
    default: Fetched 4,504 kB in 7s (611 kB/s)
    default: Reading package lists...
    default: + apt-get -y install lsb-release apt-transport-https gnupg
    default: Reading package lists...

Docker:

    default: + apt-get -dy install lsb-release apt-transport-https gnupg
    default: Reading package lists...
    default: Building dependency tree...
    default:
    default: Reading state information...
    default: Package gnupg is not available, but is referred to by another package.
    default: This may mean that the package is missing, has been obsoleted, or
    default: is only available from another source
    default: E: Package 'gnupg' has no installation candidate
    default: + apt-get update
    […]
    default: Fetched 16.2 MB in 5s (3,326 kB/s)
    default: Reading package lists...
    default: + apt-get -y install lsb-release apt-transport-https gnupg
    default: Reading package lists...

(All in green.)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 15:25:00 -07:00
YashRE42 04d9d1d27a info-overlay: Fix "message formating" styling.
The introduction of ".rendered_markdown table" in 00eaf3a, caused some
styling glitches in the "message formatting" overlay, such as an extra
right border and darker internal borders. This commit fixes these
glitches by duplicating bootstrap's table styles.
2019-05-27 14:39:09 -07:00