Commit Graph

40 Commits

Author SHA1 Message Date
Mohit Gupta 44d68c1840 refactor: Rename bugdown words to markdown in stats related functions.
This commit is part of series of commits aimed at renaming bugdown to
markdown.
2020-06-26 17:20:40 -07:00
Tim Abbott 3b4a9f8a9c test_middleware: Remove night mode test.
We removed the quoted string this test was for.
2020-06-21 01:17:42 -07:00
MariaGkoulta b10f156250 settings: Add automatic theme detection feature.
With this implementation of the feature of the automatic theme
detection, we make the following changes in the backend, frontend and
documentation.

This replaces the previous night_mode boolean with an enum, with the
default value being to use the prefers-color-scheme feature of the
operating system to determine which theme to use.

Fixes: #14451.

Co-authored-by: @kPerikou <44238834+kPerikou@users.noreply.github.com>
2020-06-21 01:09:01 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg 67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg 9824405842 dev_settings: Set ZULIP_ADMINISTRATOR.
Fixes this error in the dev environment:

$ ./manage.py checkconfig
Error: You must set ZULIP_ADMINISTRATOR in /etc/zulip/settings.py.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-09 22:15:34 -07:00
Anders Kaseorg 95f29b5409 tests: Avoid deprecated TestCase method aliases.
Fixes these warnings with python -Wd:

/home/circleci/zulip/zerver/tests/test_middleware.py:43: DeprecationWarning: Please use assertRegex instead.
  self.assertRegexpMatches(
/home/circleci/zulip/zerver/webhooks/travis/tests.py:40: DeprecationWarning: Please use assertNotEqual instead.
  self.assertNotEquals(msg.topic_name(), self.TOPIC)

https://docs.python.org/3/library/unittest.html#deprecated-aliases

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-04 14:15:52 -07:00
Mateusz Mandera dd40649e04 queue_processors: Remove the slow_queries queue.
While this functionality to post slow queries to a Zulip stream was
very useful in the early days of Zulip, when there were only a few
hundred accounts, it's long since been useless since (1) the total
request volume on larger Zulip servers run by Zulip developers, and
(2) other server operators don't want real-time notifications of slow
backend queries.  The right structure for this is just a log file.

We get rid of the queue and replace it with a "zulip.slow_queries"
logger, which will still log to /var/log/zulip/slow_queries.log for
ease of access to this information and propagate to the other logging
handlers.  Reducing the amount of queues is good for lowering zulip's
memory footprint and restart performance, since we run at least one
dedicated queue worker process for each one in most configurations.
2020-05-11 00:45:13 -07:00
Udit107710 16218d6de3 streams: Remove dependency of streams on actions.
Refactored code in actions.py and streams.py to move stream related
functions into streams.py and remove the dependency on actions.py.

validate_sender_can_write_to_stream function in actions.py was renamed
to access_stream_for_send_message in streams.py.
2020-04-18 16:56:59 -07:00
Mateusz Mandera 0255ca9b6a middleware: Log user.id/realm.string_id instead of _email. 2020-03-09 13:54:58 -07:00
Steve Howell 2e8dec233e slow queries: Use internal_send_stream_message().
Note that while the test mocks the actual message
send, we now have a `get_stream` call in the queue
worker, so we have to set up a real stream for
testing (or we could have mocked that as well, but
it didn't seem necessary).  The setup queries add
to the amount of queries reported by the test,
plus the `get_stream` call.  I just made the
query count a digits regex, which is a little bit
lame, but I don't think it's worth risking test
flakes for this.
2020-02-11 12:20:54 -08:00
Puneeth Chaganti bdf8183f55 open graph: Uploaded realm icon URL is absolute for some backends.
The `LocalUploadBackend` returns a relative URL, while the `S3UploadBackend`
returns an absolute URL. This commit switches to using `urljoin` to obtain the
absolute URL, instead of simply joining strings.
2019-05-20 13:14:28 -07:00
Anders Kaseorg cdd09d0441 meta_tags: Remove twitter:title, twitter:description, twitter:image.
Twitter falls back to og:title, og:description, og:image, which we set
identically, so these are redundant.

https://developer.twitter.com/en/docs/tweets/optimize-with-cards/overview/markup.html

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-16 21:19:08 -07:00
Puneeth Chaganti dc1571426e open graph: Use the complete URL for open graph URLs.
Closes #12199
2019-05-05 17:51:53 -07:00
Puneeth Chaganti bfc3e3c0c7 html_to_text: Add delimiters between text from different elements.
This module is used to render the HTML of pages like our user documentation 
into text for use in open graph previews of those articles.  It provided somewhat
confusing output in the case that there were paragraph breaks in the original message,
because text with multiple paragraphs and list items does't read very well. This commit
adds `|` as a delimiter between paragraphs, and prefixes list items with a `*`.

Closes #12228
2019-05-01 17:35:20 -07:00
Puneeth Chaganti f3f172991b open graph: Use uploaded realm icon for open graph images. 2019-04-28 22:50:27 -07:00
Puneeth Chaganti dabe267d00 html_to_text: Add arg to specify html tags for generating text.
Closes #11497
2019-04-25 15:29:16 -07:00
Puneeth Chaganti 2d9c5b3575 open graph: Display realm description as open graph description. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti a653fcca93 html_to_text: Escape text when using as description. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti 009beabe53 test_middleware: Use test helper in nonexistent page open graph test. 2019-04-25 15:29:16 -07:00
Puneeth Chaganti 41ff94bdba test_middleware: Parse html in tests instead of using regex.
This slows down the tests by about 5-10% -- the tests go from 0.6s to 0.630s or
so. But, this seems like a change worth making to prevent open-graph metadata
breaking HTML.
2019-04-25 15:29:16 -07:00
Anders Kaseorg 21dc34cc52 open graph: HTML-escape og:description, twitter:description.
The entire idea of doing this operation with unchecked string
replacement in a middleware class is in my opinion extremely
ill-conceived, but this fixes the most pressing problem with it
generating invalid HTML.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:53:59 -07:00
Rishi Gupta 028874bab3 open graph: Remove extraneous spaces from descriptions.
Our html collects extra spaces in a couple of places. The most prominent is
paragraphs that look like the following in the .md file:
* some text
  continued

The html will have two spaces before "continued".
2019-02-11 12:05:19 -08:00
Rishi Gupta 0119b383a6 help: Update Accounts basics section to conform to new tabbed style. 2019-02-11 12:05:19 -08:00
Rishi Gupta d3125f59e1 open graph: Omit .code-section navigation from open graph. 2019-02-11 12:05:19 -08:00
Rishi Gupta e1f02dc6f2 open graph: Include multiple paragraphs in description tags. 2019-02-11 12:05:19 -08:00
Anders Kaseorg 3127fb4dbd zerver/tests: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:43:03 -08:00
Tim Abbott 9c3f38a564 docs: Automatically construct OpenAPI metadata for help center.
This is somewhat hacky, in that in order to do what we're doing, we
need to parse the HTML of the rendered page to extract the first
paragraph to include in the open graph description field.  But
BeautifulSoup does a good job of it.

This carries a nontrivial performance penalty for loading these pages,
but overall /help/ is a low-traffic site compared to the main app, so
it doesn't matter much.

(As a sidenote, it wouldn't be a bad idea to cache this stuff).

There's lots of things we can improve in this, largely through editing
the articles, but we can deal with that over time.

Thanks to Rishi for writing all the tests.
2018-12-19 10:18:20 -08:00
Tim Abbott 7ea5987e5d errors: Use a setting to control the stream for slow-query logs.
We already had a setting for whether these logs were enabled; now it
also controls which stream the messages go to.

As part of this migration, we disable the feature in dev/production by
default; it's not useful for most environments.

Fixes the proximal data-export issue reported in #10078 (namely, a
stream with nobody ever subscribed to having been created).
2018-07-30 17:40:20 -07:00
Anders Kaseorg 037f696d26 Enable pycodestyle W605 (invalid escape sequence).
The only changes visible at the AST level, checked using
https://github.com/asottile/astpretty, are

zerver/lib/test_fixtures.py:
'\x1b\\[(1|0)m' ↦ '\\x1b\\[(1|0)m'
'\\[[X| ]\\] (\\d+_.+)\n' ↦ '\\[[X| ]\\] (\\d+_.+)\\n'

which is fine because re treats '\\x1b' and '\\n' the same way as
'\x1b' and '\n'.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Sampriti Panda 3f4200db3c tests: Disable slow query messages in test environment.
Slow queries during backend tests sends messages to Error Bot
which affects the database state causing the tests to fail.
This fixes the occasional flakes due to that.
2018-05-20 10:16:53 -07:00
rht 6c28c7e879 zerver/tests: Use python 3 syntax for typing. 2017-11-18 15:35:57 -08:00
rht 4f5b1c0a5a zerver/tests: Use python 3 syntax for typing in most files. 2017-11-16 21:52:01 -08:00
derAnfaenger 7777a81dbf queue_processors: Add full coverage for SlowQuery. 2017-11-10 13:23:53 -08:00
derAnfaenger 3ac09b3e9b queue processors: Add coverage for SlowQueryWorker. 2017-11-09 15:20:40 -08:00
Alena Volkova 5515a075ec urls: Move the report endpoints to be API-style routes. 2017-10-17 22:05:56 -07:00
rht 26f5d9a32c zerver/tests: Remove print_function. 2017-09-27 18:05:45 -07:00
rht 1e87a4b68c zerver/tests: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott 94ed405c69 tests: Extract zerver/tests/test_middleware.py. 2017-03-08 03:57:37 -08:00