Extracting fields from a url in scala-js

Question

Suppose I've a url like :

https://example.com/myproject/index-dev.html?_ijt=hsdlgh8h5g8hh489sajoej&a=102&b=a%20m&c=45&d=all&e=all

or it may be a webpage on localhost like :

localhost:63342/my project/index-dev.html?_ijt=hsdlgh8h5g8hh489sajoej&a=102&b=a%20m&c=45&d=all&e=all

and I've to extract query fields (appearing after '?') from these urls in 2-D array as following :

_ijt    |    hsdlgh8h5g8hh489sajoej
a       |    102
b       |    a m
c       |    45
d       |    all
e       |    all

Please do note that in 'b' field, I've replaced '%20' with a space. These fields like _ijt,a,b,c,d,e etc can vary in number and their names eg 'a' can be 'city'. So far I've used regular expression to extract out the part after '?' and then use split("&") method to split the string into multiple strings. Code -

val url=localhost:63342/my project/index-dev.html?_ijt=hsdlgh8h5g8hh489sajoej&a=102&b=a%20m&c=45&d=all&e=all
val pattern="""(http|htpps)([A-Za-z0-9\:\/\%\-\.]*)\?""".r
val temp_url=pattern.replaceFirstIn(url,"")
val fields=temp_url.split("&")
println(fields.foreach(println))

and the output is :

_ijt=hsdlgh8h5g8hh489sajoej
a=102
b=a%20m
c=45
d=all
e=all

But it doesn't seem to be the correct way to do this. Any help ?

sjrd · Accepted Answer · 2016-12-06 07:17:01Z

5

Use js.URIUtils.decodeURIComponent to accurately decode the %-encoded characters.

answered Dec 6, 2016 at 7:17

sjrd

22.2k2 gold badges65 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ishan Over a year ago

It solves the issue in 'a%20m' but suppose at the end of the url, I add #clinic then it will show engine=all#clinic which is undesirable. Further I've to use fields.foreach(println) to print those fields. But I want to store each field into a separate variable. How can I achieve this ?

sjrd Over a year ago

Use java.net.URI to reliably parse a URL like that. It is supported by Scala.js (java.net.URL is not, though).

Nikita · Accepted Answer · 2016-12-07 06:21:50Z

2

You need to call js.URIUtils.decodeURIComponent on query parameter values:

val fields=temp_url.split("&").map(js.URIUtils.decodeURIComponent)

decodeURIComponent is a native Javascript function, for which scala.js has a simple interface.

Alternatively, you could use some library for parsing URLs written in Scala. Parsing URLs is often a security hazard, and it's easy to make a mistake. Libraries also typically support any input that satisfies the relevant Standards / RFCs.

edited Dec 7, 2016 at 6:21

answered Dec 6, 2016 at 7:14

Nikita

2,9631 gold badge22 silver badges26 bronze badges

Comments

James Moore · Accepted Answer · 2017-08-02 14:30:27Z

Surprisingly, it's hard to find a library for the whole thing that works both in Scala.js and Scala(jvm). java.net.URI will get you the query string, and java.net.URLDecoder.decode will remove the URL encoding, but I haven't seen anything that gets you nicely structured query pieces. It's not rocket science, but it's so common that you'd think you wouldn't need to write it yourself.

Fastparse will do the job:

  val url = new java.net.URI("http://example.com/?a=1&b%20=b+is+2&c=#someAnchor?a=b")
  println(s"query string is: ${url.getQuery}")

  val individualElements =
    P(CharsWhile {
      case '&' | '=' | '#' => false
      case _ => true
    }.!.map(x => java.net.URLDecoder.decode(x, "UTF-8")))

  val keyValuePair: core.Parser[(String, Option[String]), Char, String] =
    individualElements ~ "=" ~ individualElements.?

  val pairs: core.Parser[Seq[(String, Option[String])], Char, String] =
    keyValuePair.rep(sep = "&")

  val parsed: Parsed[Seq[(String, Option[String])], Char, String] =
    pairs.parse(url.getQuery)

  parsed match {
    case Success(items, _) => println(s"items: ${items.toList}")
    // prints:
    // items: List((a,Some(1)), (b ,Some(b is 2)), (c,None))
  }

Adam Lane · Accepted Answer · 2017-11-17 18:34:21Z

0

Building on the suggestions to use "decodeURIComponent" and "java.net.URI" I came up with this quick and dirty solution that almost certainly could be improved but perhaps it helps:

def getUrlParameters(url: String): Map[String, Array[String]] = {
    java.net.URI.create(url).getQuery.split('&').map(js.URIUtils.decodeURIComponent).map { p =>
      val split = p.split('=')
      (split.head, split.tail.mkString("="))
    }.groupBy(_._1).map(m => m._1 -> m._2.map(_._2))
}

def getUrlParameter(url: String, parameter: String): Option[String] = {
    getUrlParameters(url).get(parameter).flatMap(_.headOption)
}

answered Nov 17, 2017 at 18:34

Adam Lane

1,92422 silver badges26 bronze badges

Comments

shvahabi · Accepted Answer · 2020-05-03 22:11:37Z

0

A bit too late, however out of hope, to be helpful to anybody come here from google; let:

def uriParameterExtractor(uri: String): Map[String, String] =
js.URIUtils.decodeURIComponent(uri).tail.split("&").toList.map(x => (x.split("=").head, x.split("=").tail.head)).toMap

then call this function with dom.window.location.search argument. You can query any parameter value of interest by using respective key on result of this function, i.e:

uriParameterExtractor(dom.window.location.search)("a") will evaluate to 102
uriParameterExtractor(dom.window.location.search)("b") will evaluate to a m
and etc.

answered May 3, 2020 at 22:11

shvahabi

1,31810 silver badges9 bronze badges

Comments

David Portabella · Accepted Answer · 2020-05-31 18:14:19Z

-1

Use this scala / scalajs library: https://github.com/lemonlabsuk/scala-uri

import io.lemonlabs.uri.Url

val uri = Url.parse("http://example.com/path?a=b&a=c&d=e")
uri.query.paramMap // This is: Map("a" -> Vector("b", "c"), "d" -> Vector("e"))

edited May 31, 2020 at 18:14

answered May 31, 2020 at 17:56

David Portabella

12.8k29 gold badges106 silver badges195 bronze badges

Collectives™ on Stack Overflow

Extracting fields from a url in scala-js

6 Answers 6

2 Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

2 Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related